Information-signal process apparatus and information-signal processing method

ABSTRACT

A recording/reproduction apparatus that effectively and efficiently utilizes a large-quantity information signal recorded onto a recording medium and efficiently utilizing the recording medium for recording the information signal. In a process to record a video signal onto the recording medium, a video classification process unit employed in the recording/reproduction apparatus recognizes similar pictures of the video signal, classifies pictures of the signal into similar-picture segments each including similar pictures and classifies the similar-picture segments into groups each including similar-picture segments resembling each other. Then, the recording/reproduction apparatus displays a picture as a thumbnail picture for each of the similar-picture segments on a classification display unit.

TECHNICAL FIELD

The present invention typically relates to an information-signal processapparatus for recording information signals such as audio and videosignals onto a recording medium and an information-signal process methodadopted by such an information-signal process apparatus as well asrelates to an information-signal process apparatus for reproducing aninformation signal from a recording medium and an information-signalprocess method adopted by such an information-signal process apparatus.

BACKGROUND ART

In recent years, a large number of various video and audio signals ispresented not only through the conventional ground-wave broadcasting butalso through a variety of communication routes such as BS (BroadcastingSatellite) digital broadcasting and communication networks. Speakingabout the communication networks, a large number of digital video andaudio signals is presented to the end user at a high speed.

In addition, the storage capacities of a variety of recording mediums orthe storage capacities of various kinds of recording media areincreasing at a very high pace and, at the same time, the prices of therecording mediums are decreasing in a short period of time. Therecording mediums include a magnetic disk such as a hard disk, anoptical disk such as a DVD (Digital Versatile Disk) and amagneto-optical disk known as an MO disk. Thus, the end user can store alarge number of information signals such as video and audio signalspresented to the user through a variety of communication routes in arecording medium owned by the user itself by using arecording/reproduction apparatus also of its own as a large amount ofdata and use the signals.

By the way, if information signals such as video and audio signals of abroadcast program have been stored in a disk-shaped recording mediumsuch as a hard disk having a large storage capacity as described above,a conceivably rising demand for effective and efficient utilization ofthe information signals can be met by taking advantage of theaccessibility merit, which is offered by the disk-shaped recordingmedium to allow the medium to be searched for a desired scene of adesired broadcast program at a high speed so as to allow the scene to bereproduced quickly.

Even though the storage capacity of the recording medium is large, thecapacity is by no means unlimited, requiring considerations to utilizethe recording medium as efficiently as possible and to avoid a problemcaused by an insufficient storage capacity.

It is thus an object of the present invention addressing the aboveproblems to provide an information-signal process apparatus capable ofrecording a large number of information signals effectively and with ahigh degree of efficiency onto a recording medium having a large storagecapacity for storing the information signals and capable of utilizingthe recording medium with a high degree of efficiency as well as providean information-signal process method to be adopted by theinformation-signal process apparatus.

DISCLOSURE OF INVENTION

In a process to record video and audio signals of a television broadcastprogram or the like onto a recording medium, a recording/reproductionapparatus provided by the present invention detects information oncharacteristics of the video signal, carries out a similaritydetermination process based on the detected characteristics and carriesout a process to classify pictures into similar-picture segments eachconsisting of similar pictures.

In a recording/reproduction apparatus implemented by another embodimentof the present invention, principal information and summary informationare managed by associating the principal information and the summaryinformation with each other. When the storage size of a recording arearemaining in a recording medium becomes insufficient, the summaryinformation is deliberately left in the recording medium, takingprecedence of the principal information, so that the recording mediumcan be utilized effectively and with a high degree of efficiency and, inaddition, it is possible to efficiently and effectively make use of aninformation signal such as a video signal remaining on the recordingmedium.

On top of that, in a recording/reproduction apparatus implemented by afurther embodiment of the present invention, when the storage size of arecording area remaining in a first recording medium becomes smallerthan a predetermined value, least recent principal information is movedto a second recording medium to increase the storage size of therecording area remaining in the first recording medium and, hence,allocate a larger residual recording area in the first recording medium,before recording new principal information onto the first recordingmedium. Thus, it is possible to avoid a problem such as an inability tostore principal information of a new program due to an insufficientresidual storage capacity of the first recording medium.

Furthermore, in a recording/reproduction apparatus implemented by astill further embodiment of the present invention, in a process torecord video and audio signals of a plurality of television broadcastprograms onto a recording medium, similar-picture segments areidentified, pictures located at the heads of the similar-picturesegments are displayed as shrunk pictures (or thumb-nail pictures) in atabular format (a multi-picture format) and the display in the tabularformat is updated from time to time. Thus, contents of the televisionbroadcast programs recorded on the recording medium can be graspedeffectively and with a high degree of efficiency even if the display ofshrunk pictures varies with the lapse of time.

Moreover, in a recording/reproduction apparatus implemented by a stillfurther embodiment of the present invention, an effective digestreproduction mode to be used for reproducing a program from a recordingmedium is selected in accordance with the genre of the program or inaccordance with a command entered by the user so that an effective aswell as efficient digest reproduction process can be carried out.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory block diagram showing a recording/reproductionapparatus implemented by a first embodiment to which an embodiment of aninformation-signal process apparatus and information-signal processmethod provided by the present invention is applied;

FIGS. 2A to 2C are explanatory diagrams showing an outline of operationscarried out by the recording/reproduction apparatus implemented by thefirst embodiment shown in FIG. 1;

FIG. 3 is an explanatory diagram showing an outline of operationscarried out by the recording/reproduction apparatus implemented by thefirst embodiment shown in FIG. 1;

FIG. 4 is an explanatory block diagram showing a picture classificationprocess unit employed in the recording/reproduction apparatusimplemented by the first embodiment shown in FIG. 1;

FIG. 5 is a diagram showing operations carried out by the pictureclassification process unit employed in the recording/reproductionapparatus implemented by the first embodiment shown in FIG. 1;

FIG. 6 is a diagram showing operations carried out by the pictureclassification process unit employed in the recording/reproductionapparatus implemented by the first embodiment shown in FIG. 1;

FIG. 7 is an explanatory block diagram showing a sound classificationprocess unit employed in the recording/reproduction apparatusimplemented by the first embodiment shown in FIG. 1;

FIG. 8 shows a flowchart referred to in explanation of operationscarried out by the recording/reproduction apparatus implemented by thefirst embodiment shown in FIG. 1;

FIG. 9 shows a continuation flowchart of the flowchart shown in FIG. 8;

FIG. 10 shows a flowchart referred to in explanation of other operationscarried out by the recording/reproduction apparatus implemented by thefirst embodiment shown in FIG. 1;

FIG. 11 shows a continuation flowchart of the flowchart shown in FIG.10;

FIG. 12 is an explanatory block diagram showing a recording/reproductionapparatus implemented by a second embodiment to which an embodiment ofthe information-signal process apparatus and information-signal processmethod provided by the present invention is applied;

FIGS. 13A and 13B are explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the second embodiment shown in FIG. 12;

FIGS. 14A to 14B are an explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the second embodiment shown in FIG. 12;

FIGS. 15A to 15B are an explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the second embodiment shown in FIG. 12;

FIG. 16 shows a flowchart referred to in explanation of operationscarried out by the recording/reproduction apparatus implemented by thesecond embodiment shown in FIG. 12;

FIG. 17 shows a continuation flowchart of the flowchart shown in FIG.16;

FIG. 18 shows a flowchart referred to in explanation of other operationscarried out by the recording/reproduction apparatus implemented by thesecond embodiment shown in FIG. 12;

FIG. 19 is an explanatory block diagram showing a recording/reproductionapparatus implemented by a third embodiment to which an embodiment ofthe information-signal process apparatus and information-signal processmethod provided by the present invention is applied;

FIG. 20 shows a flowchart referred to in explanation of operationscarried out by the recording/reproduction apparatus implemented by thethird embodiment shown in FIG. 19;

FIG. 21 shows a continuation flowchart of the flowchart shown in FIG.20;

FIG. 22 shows another continuation flowchart of the flowchart shown inFIG. 20;

FIG. 23 is an explanatory block diagram showing a recording/reproductionapparatus implemented by a fourth embodiment to which an embodiment ofthe information-signal process apparatus and information-signal processmethod provided by the present invention is applied;

FIGS. 24A to 24B are an explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the fourth embodiment shown in FIG. 23;

FIGS. 25A to 25C are explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the fourth embodiment shown in FIG. 23;

FIG. 26 shows a flowchart referred to in explanation of operationscarried out by the recording/reproduction apparatus implemented by thefourth embodiment shown in FIG. 23;

FIG. 27 is an explanatory block diagram showing a recording/reproductionapparatus implemented by fifth and sixth embodiments to which anembodiment of the information-signal process apparatus andinformation-signal process method provided by the present invention isapplied;

FIGS. 28A to 28C are explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the fifth embodiment shown in FIG. 27;

FIGS. 29A to 29C are explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the fifth embodiment shown in FIG. 27;

FIG. 30 shows a flowchart referred to in explanation of operationscarried out by the recording/reproduction apparatus implemented by thefifth embodiment shown in FIG. 27;

FIG. 31 shows a continuation flowchart of the flowchart shown in FIG.30;

FIG. 32 shows a continuation flowchart of the flowchart shown in FIG.31;

FIGS. 33A and 33B are explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the sixth embodiment shown in FIG. 27;

FIGS. 34A to 34C are explanatory diagrams showing an outline ofoperations carried out by the recording/reproduction apparatusimplemented by the sixth embodiment shown in FIG. 27;

FIG. 35 shows a flowchart referred to in explanation of operationscarried out by the recording/reproduction apparatus implemented by thesixth embodiment shown in FIG. 27; and

FIG. 36 shows a continuation flowchart of the flowchart shown in FIG.35.

BEST MODES FOR CARRYING OUT THE INVENTION

By referring to diagrams, the following description explains embodimentsimplementing an information-signal process apparatus and aninformation-signal process method, which are provided by the presentinvention. The embodiments described below each implement theinformation-signal process apparatus and information-signal processmethod provided by the present invention in their typical applicationsto a recording/reproduction apparatus for recording video and audiosignals of a television broadcast program onto a recording medium andreproducing video and audio signals from a recording medium.

First Embodiment

FIG. 1 is an explanatory block diagram showing a recording/reproductionapparatus 100 implemented by a first embodiment to which an embodimentof an information-signal process apparatus and information-signalprocess method provided by the present invention is applied. FIGS. 2A to2C and 3 are explanatory diagrams showing the principle of operation ofthe recording/reproduction apparatus 100 implemented by this embodiment.

The recording/reproduction apparatus 100 implemented by the firstembodiment is capable of playing back an information signal of typicallya television broadcast program from a recording medium having goodaccessibility and a large storage capacity without reproducing alldetails of the signal so as to allow recorded contents of the signal tobe grasped quickly, simply and accurately, hence, promoting effectiveand efficient use of the recorded signal. An example of such a recordingmedium is a hard disk.

<Principle of Operation>

First of all, the principle of operation of the recording/reproductionapparatus 100 implemented by the first embodiment is explained. Therecording/reproduction apparatus 100 shown in FIG. 1 is capable ofreceiving an analog video and audio signals of typically a televisionbroadcast program as information signals, converting the signals intodigital data and compressing the digital data by adopting an MPEG(Moving Picture Experts Group) method. The recording/reproductionapparatus 100 is also capable of multiplexing the compressed video dataand the compressed audio data on a time-division basis before recordingthe multiplexed compressed data onto a recording medium 8. The recordingmedium 8 is typically a disk-shaped recording medium having a largestorage capacity. An example of such a disk-shaped recording medium is ahard disk.

In addition, the recording/reproduction apparatus 100 shown in FIG. 1 isalso capable of reading out a multiplexed compressed signal from therecording medium 8 and demultiplexing the multiplexed compressed signalread out from the recording medium 8 to generate compressed video dataand audio data. The recording/reproduction apparatus 100 is also capableof decompressing the compressed video and audio data to restorepre-compression video and audio data respectively as well as convertingthe pre-compression video and audio data into analog signals.

In the process to record video and audio signals onto the recordingmedium 8, the recording/reproduction apparatus 100 shown in FIG. 1carries out processing to classify the video signal into segments eachconsisting of similar pictures. The classification processing is appliedto compressed video data, which is referred to as MPEG data. It is to benoted that, since a pictorial image can be assumed to be an image notchanging frequently from frame to frame, the classification processingis carried out by detecting similarity only for each I (Intra-code)picture (frame) to classify frames into segments each consisting ofsimilar pictures or frames. The classification processing is thuscarried out to identify similar-picture segments each consisting ofconsecutive similar frames and to identify similar-picture segments thatresemble each other.

As will also be described later, the video classification process unit 5employed in the recording/reproduction apparatus 100 shown in FIG. 1carries out processing to detect similar pictures and processing toclassify the detected similar pictures into similar-picture segments.Thus, each similar-picture segment including consecutive similarpictures can be identified and it is also possible to determinesimilar-picture segments resembling each other.

As an example, assume an input picture signal series consisting of 31successive pictures as shown in FIG. 2A. This input picture signalseries is subjected to the classification processing to identifysimilar-picture segments A1, A2, A3, B1, B2 and B3, in each of which Ipictures are similar to each other. On the other hand, theclassification processing also detects segments C1 and C2 asnon-similar-picture segments, in each of which I pictures are notsimilar to each other.

In addition, the recording/reproduction apparatus 100 shown in FIG. 1also has a function capable of recognizing an I picture not similar toan immediately preceding I picture as an I picture located at ascene-change point. By executing this function, therecording/reproduction apparatus 100 is capable of detecting I pictureseach pointed to by an arrow in FIG. 2A as I pictures each located at ascene-change point. That is to say, the 1^(st), 4^(th), 5^(th), 6^(th),7^(th), 10^(th), 11^(th), 12^(th), 17^(th), 21^(st) and 29^(th) Ipictures are each detected as an I picture located at a scene-changepoint.

Moreover, the classification processing is carried out also to furtherclassify similar-picture segments in order to put similar-picturesegments resembling each other in a group of similar-picture segmentsresembling each other. In the case of this input picture signal series,the similar-picture segments A1, A2 and A3 are put in a group ofsimilar-picture segments resembling each other, the similar-picturesegments B1, B2 and B3 are put in another group of similar-picturesegments resembling each other and the similar-picture segments C1 andC2 are put in a further group of similar-picture segments resemblingeach other.

Thus, when the recording/reproduction apparatus 100 implemented by thisembodiment receives the input picture signal series shown in FIG. 2A,the recording/reproduction apparatus 100 classifies the similar-picturesegments included in the input picture signal series into 3 groups ofsimilar-picture segments resembling each other, i.e., the first groupconsisting the similar-picture segments A1, A2 and A3 resembling eachother, the second group consisting the similar-picture segments B1, B2and B3 resembling each other and the third group consisting thenon-similar-picture segments C1 and C2 resembling each other. Therecording/reproduction apparatus 100 also finds how many I pictures areincluded in each of the similar-picture segments composing each of thegroups.

In the process to record an information signal onto the recording medium8, as described above, the recording/reproduction apparatus 100implemented by this embodiment shown in FIG. 1 carries out theprocessing to detect similarity among pictures and classify the picturesinto similar-picture segments. Then, with a predetermined timing such asthe timing of a recording-process end, a result of the classification ofthe similar-picture segments is displayed as shown in FIG. 3.

In this case, the result of the classification is displayed for eachgroup of similar-picture segments resembling each other. A result of theclassification for a group of similar-picture segments resembling eachother is displayed on a row as an array of shrunk forms of I pictureseach positioned at the head of one of the similar-picture segments. Toput it concretely, the display begins with an array having a highestsimilarity detection frequency or an array accommodating a largestnumber of similar I pictures. In addition, every array starts with an Ipicture having a smallest frame number among the I pictures accommodatedin the array to be followed by I pictures arranged in an order ofincreasing frame numbers. The I pictures displayed in every array areeach a shrunk form of a picture located at a change-over point in thegroup represented by the array or a scene-change point detected by theclassification process.

Note that it is necessary to pay attention to the fact that asimilar-picture segment with a camera pan has picture scenes changingwith the camera pan. Thus, when taking such a similar-picture segmentinto consideration from the picture-similarity point of view, thepicture scenes are considered to be pictures not similar to each other.With a camera pan merely existing, on the other hand, the segment canalso be considered to be a segment not including a cut point (or a scenechange). That is to say, there is no absolute standard usable fordetermining whether or not certain pictures are similar to each otherand a change-over point of time in a group may not necessarily be thesame as a scene-change point of time.

In addition, in the example described above, if two or more groups eachconsisting of similar-picture segments resembling each other have equalsimilarity detection frequencies, the group starting with an I picturehaving the smallest frame number is displayed first to be followed by agroup starting with an I picture having a larger frame number.

In the typical display shown in FIG. 3, the screen is divided into amatrix of 8×8 small screen units used for displaying only shrunk formsof pictures each located at an scene-change point in the array of Ipictures obtained as a result of a classification process as shown inFIG. 2A. To be more specific, the small screen units on each horizontalrow of the matrix are used for displaying shrunk forms of pictureslocated at scene-change points at the heads of similar-picture segmentsincluded in a group of the similar-picture segments resembling eachother, and the small screen units on each vertical column of the matrixare used for displaying shrunk forms of pictures each located at anscene-change point at the head of every similar-picture segment.

Thus, to put it concretely, in the typical display shown in FIG. 3, thetop horizontal row c1 is used for displaying shrunk forms of the 1^(st),12^(th) and 21^(st) I pictures located at scene-change points at theheads of the similar-picture segments A1, A2 and A3 respectively.

By the same token, the second row c2 is used for displaying shrunk formsof the 7^(th), 17^(th) and 29^(th) I pictures located at scene-changepoints at the heads of the similar-picture segments B1, B2 and B3respectively. In the same way, the horizontal row c3 is used fordisplaying shrunk forms of the 4^(th) and 10^(th) I pictures located atscene-change points at the heads of the non-similar-picture segments C1and C2 resembling each other.

As described above, the recording/reproduction apparatus 100 implementedby the first embodiment carries out the processing to detect similarityamong pictures and classify the pictures into similar-picture segmentsin the recording process and displays a result of the picturesclassification processing at a recording-execution time, at arecording-end time or with a proper timing after the end of therecording process. Thus, it is possible to quickly and reliably knowwhat broadcast program has been recorded onto the recording medium. Thatis to say, the contents of the recorded broadcast program can be graspedquickly and reliably without reproducing all details of the recordedinformation signal and, hence, the information signal can be utilizedeffectively and efficiently.

In addition, in order to provide a tabular display (or a multi-picturedisplay) showing a result of classification as shown in FIG. 3, shrunkpictures (or thumb-nail pictures) are required. FIG. 2B is a diagramshowing an array of shrunk pictures arranged in the same time sequenceshown in FIG. 2A as a time sequence of the I pictures with their datacompressed by using the MPEG method. By storing this sequence of shrunkpictures in the recording medium in advance, a tabular display of theshrunk pictures (or the thumb-nail pictures) can be provided immediatelyafter the classification processing.

Here, consider a case in which shrunk pictures are displayed as shown inFIG. 3 at the recording-end time. In this case, during the recordingprocess, the shrunk pictures are created and stored onto the recordingmedium along with the original picture data to shorten thesignal-processing time. If the shrunk pictures are not created in theprocess to record the original pictures, both a process to decode theoriginal data and a process to create the shrunk pictures must becarried out. This is because the data of the original pictures iscompressed data. Thus, the shrunk pictures cannot be displayed quickly.

In order to solve this problem, the shrunk pictures are created alongwith the original picture data as shown in FIG. 2B and stored onto therecording medium along with the original picture data to shorten thesignal-processing time. In addition, each of the original pictures,which are I pictures, can be associated easily with a shrunk picturethereof.

If the shrunk pictures shown in FIG. 2B are all stored in the recordingmedium, however, the amount of shrunk-picture data is large so thatthere may be conceivably a case in which the recording medium having alimited storage capacity can no longer be utilized effectively due tothe large amount of shrunk-picture data. In order to solve this problem,the recording/reproduction apparatus 100 implemented by the firstembodiment shown in FIG. 1 stores only shrunk pictures located atscene-change points by associating the shrunk pictures with the originalpictures each located at the scene-change point in order to reduce theamount of shrunk-picture data to a minimum.

Now, consider the amount of data of shrunk pictures in a base-bandregion for which the tabular display (or the multi-picture display)shown in FIG. 3 is expressed in terms of “0” and “1”. In the case of thetypical display shown in FIG. 3, the data of a shrunk picture has a sizeequal to ⅛ times (one-eighth) of the size of the original picturecorresponding to the shrunk picture. That is to say, the height(vertical dimension) and width (horizontal dimension) of a shrunkpicture are ⅛ times the vertical and horizontal dimensions of theoriginal picture.

Let the data of each original picture be expressed in terms of a Ysignal (luminance signal), a Cb signal (blue color-difference signal)and a Cr signal (red color-difference signal), which each have a size of8 bits, and the resolution of each original picture be 352 (horizontaldirection)×240 (vertical direction). In this case, the amount of dataper shrunk picture can be expressed in accordance with Eq. (1) asfollows:Amount of data per shrunk picture=3×8 bits×(352×240)×(⅛)×(⅛)=31,680bits≈4 Kbyte  (1)

Assume for example that the recording time of a broadcast program is twohours, the program comprises 30 frames per second, being compressed intoan MPEG stream composed of 15 pictures per GOP (group of pictures). Inthis case, the amount of data of all shrunk pictures can be expressed inaccordance with Eq. (2) as follows:Amount of data of all shrunk pictures=2H×60 min/H×60 sec/min×31, 680bit×30 frames/sec×( 1/15)=54 Mbyte  (2)

As is obvious from the above equation, even the amount of data of allshrunk pictures is large.

However, pictures included in any similar-picture segment shown in FIG.2A are similar to each other. Thus, for every similar-picture segment,it is necessary to generate only a shrunk picture of the originalpicture located at the head of the similar-picture segment, that is, ashrunk picture of the original picture located at a scene-change pointof the similar-picture segment. By storing only such shrunk picturesalong with the original pictures, the amount of data of the shrunkpictures can be reduced.

For example, since the 1^(st), 2^(nd) and 3^(rd) original pictures inthe similar-picture segment A1 are similar to each other, only theshrunk picture of the 1^(st) original picture located at thescene-change point of the similar-picture segment A1 is created andstored in the recording medium. The technique of selecting an originalpicture for which a shrunk picture is to be generated is applied to theother similar-picture segments A2, A3, B1, B2 and B3.

As for the non-similar-picture segments C1 and C2, pictures includedtherein are not similar to each other. Thus, a shrunk picture is createdfor each of all original pictures included in every non-similar-picturesegment and stored in the recording medium along with the original ones.To put it concretely, in the case of the non-similar-picture segment C1,a shrunk picture is created for each of the 4^(th), 5^(th) and 6^(th)original pictures and stored in the recording medium. The technique ofselecting original pictures for each of which a shrunk picture is to begenerated is applied to the other non-similar-picture segment C2.

By selecting original pictures, for each of which a shrunk picture is tobe generated, as described above, the 1^(st), 4^(th), 5^(th), 6^(th),7^(th), 10^(th), 11^(th), 12^(th), 17^(th), 21^(st) and 29^(th) originalpictures shown in FIG. 2C are each selected as an original picture forwhich a shrunk picture is to be generated and the generated shrunkpictures are stored in the recording medium along with all originalpictures.

Thus, the amount of data of shrunk pictures created for originalpictures shown in FIG. 2C can be made much smaller than the amount ofdata of shrunk pictures created for original pictures shown in FIG. 2B.This is because the shrunk pictures for the array shown in FIG. 2C arebasically created only for original pictures each located at the head ofa similar-picture segment, that is, original pictures each located at ascene-change point of the similar-picture segment. If it is assumed thatthe recording time of a broadcast program is two hours as describedabove and the average duration of similar-picture segments is aboutthree seconds, the creation of shrunk pictures only for originalpictures selected by using the method to process fewer original pictureslike those shown in FIG. 2C reduces the amount of data of the shrunkpictures to about 18 Mbyte.

It is to be noted that a shrunk picture can further be compressed byadoption of a data compression technique such as the JPEG (JointPhotographic Expert Group) to further reduce the amount of data of theshrunk picture.

The recording/reproduction apparatus 100 implemented by the firstembodiment shown in FIG. 1 is capable of adopting the method to selectonly original pictures like those shown in FIG. 2B and the method toselect only original pictures like those shown in FIG. 2C. In thefollowing description, however, it is assumed that the method to selectonly original pictures like those shown in FIG. 2C is mainly adopted.

<Operations and Configuration of the Recording/Reproduction Apparatus100 Implemented by the First Embodiment>

The following description explains operations and configuration of therecording/reproduction apparatus 100 implemented by the first embodimentshown in FIG. 1. The configuration shown in FIG. 1 can be divided intoportions on the left and right sides of the recording medium 8. Theportion on the left side is mainly a recording system while the portionon the right side is mainly a reproduction system.

<Recording System>

First of all, the recording system is explained. An analog audio signal(an audio input) supplied to the recording/reproduction apparatus 100implemented by the first embodiment is fed to an audio analog-to-digitalconverter 1, which is abbreviated hereafter to an audio A/D converter,for converting the analog audio signal into a digital audio signal to besupplied to an audio encoder 2 and an audio classification process unit20.

The audio encoder 2 carries out a band compression process on thedigital audio signal by adoption of a predetermined compressiontechnique such as an MPEG audio method. Audio data obtained as a resultof the band compression process carried out on the digital audio signalis supplied to a multiplexing process unit 6.

The audio classification process unit 20 detects continuity of afrequency spectrum peak in the digital audio signal typically for eachpredetermined segment to determine whether or not the digital audiosignal is a musical signal. The result of the determination is suppliedto the multiplexing process unit 6, a system controller 17 and apicture-scene classification display and reproduction-position displayunit 19. The picture-scene classification display andreproduction-position display unit 19 is also properly referred tohereafter as a classification display unit.

It is to be noted that, instead of processing the digital audio signalreceived from the audio A/D converter 1, the audio classificationprocess unit 20 can also be designed into a configuration for receivingthe audio data, which is obtained as a result of the band compressionprocess carried out by the audio encoder 2 on the digital analog signalgenerated by the audio A/D converter 1, as indicated by a dashed lineshown in FIG. 1, and carrying out the classification processingdescribed earlier on the audio data.

On the other hand, an analog video signal (a video input) supplied tothe recording/reproduction apparatus 100 implemented by the firstembodiment is fed to a video analog-to-digital converter 3, which isabbreviated hereafter to a video A/D converter, for converting theanalog video signal into a digital video signal to be supplied to avideo encoder 4.

The video encoder 4 carries out a band compression process on thedigital video signal supplied thereto, by adoption of a predeterminedcompression technique such as an MPEG2 method. Video data obtained as aresult of the band compression process carried out on the digital videosignal is supplied to a video classification process unit 5, themultiplexing process unit 6, an I-picture detection unit 21 and ashrunk-picture process unit 22.

The video classification process unit 5 detects picture characteristicsof I pictures of the video data received from the video encoder 4 andrecognizes picture similarity on the basis of the detectedcharacteristics. In addition, the video classification process unit 5also carries out a series of picture classification processes includingidentification of similar-picture segments, non-similar-picture segmentsand groups each consisting similar-picture segments resembling eachother as explained earlier by referring to FIGS. 2A to 2C. The videoclassification process unit 5 also generates an identificationinformation signal for identifying the similar-picture segments, thenon-similar-picture segments and the groups each consistingsimilar-picture segments resembling each other, supplying the signal tothe multiplexing process unit 6 and the classification display unit 19.

The identification information signal includes position information ofthe video data. The position information of the video data includesinformation on how many similar frames are included in eachsimilar-picture segment. In addition, the video classification processunit 5 is also capable of detecting scene-change points on theprocessing of picture-similarity determination, and reporting them tothe system controller 17.

It is to be noted that, instead of receiving the compressed video datafrom the video encoder 4, the video classification process unit 5 mayalso receive the digital video signal output by the video A/D converter3 as indicated by a dashed line shown in FIG. 1 and carries out theprocesses to identify and classify similar pictures of the digital videodata in frame units as described above on the basis of the digital videosignal.

In addition, it is also possible to provide a configuration in which theidentification information signals obtained as results of the processesto classify the video and audio signals can also be generated by thesystem controller 17 on the basis of the classification-process resultsreceived from the video classification process unit 5 and the audioclassification process unit 20 and supplied to the multiplexing processunit 6.

Shown in none of the figures, the system controller 17 is amicrocomputer including a CPU (Central Processing Unit), a ROM (ReadOnly Memory), a RAM (Random Access Memory), a non-volatile memory and atimer circuit, which are used for controlling the other componentscomposing the recording/reproduction apparatus 100.

In addition, the I-picture detection unit 21 in the configuration shownin FIG. 1 is a component for detecting I pictures from the video dataand reporting detection information such as detection timings to thesystem controller 17. Only I pictures are detected because, as describedabove, the recording/reproduction apparatus 100 implemented by thisembodiment carries out the picture-similarity detection process and thepicture classification process in I-picture units. The system controller17 is capable of controlling the video classification process unit 5 andthe shrunk-picture process unit 22 on the basis of the detectioninformation received from the I-picture detection unit 21.

In accordance with the control executed by the system controller 17, theshrunk-picture process unit 22 generates shrunk-picture data from videodata (to be more specific, I pictures each located at a scene-changepoint) received thereby and supplies the data to the multiplexingprocess unit 6 and the classification display unit 19.

In this embodiment, the classification display unit 19 receives theclassification result from the audio classification process unit 20, theidentification information signal from the video classification processunit 5 and the shrunk-picture data from the shrunk-picture process unit22, displaying classifications of shrunk pictures, which are arranged onmatrix rows each corresponding to a group of similar-picture segmentsresembling each other and each represent one of the similar-picturesegments as shown in FIG. 3. That is to say, the recording/reproductionapparatus 100 implemented by this embodiment is capable of displayingthumb-nail pictures for recorded original pictures concurrently with theprocess to record the original pictures onto the recording medium.

In addition, the multiplexing process unit 6 generates multiplexed databy typically carrying out a time-division multiplexing process on theaudio data, the video data, the shrunk-picture data and theidentification information signal which is received from the videoclassification process unit 5, supplying the multiplexed data to therecording process unit 7. The recording process unit 7 carries outprocessing including a process of adding error correction codes to themultiplexed data and a process of interleaving the data to create asignal to be recorded, recording the signal into a predetermined area inthe recording medium 8.

Then, in the case of this embodiment, as described before, by using theidentification information signal multiplexed in the multiplexed data,it is possible to identify similar-picture segments, non-similar-picturesegments and groups each consisting of similar-picture segmentsresembling each other and associate original pictures with shrunkpictures representing the original pictures. Thus, it is possible todisplay classifications of shrunk pictures, which are arranged on matrixrows each corresponding to a group of similar-picture segmentsresembling each other and each represent one of the similar-picturesegments as shown in FIG. 3, during a recording process or at arecording-end time or with a predetermined timing after the end of therecording process.

<Picture Classification Processing>

The picture classification processing carried out by the videoclassification process unit 5 employed in the recording/reproductionapparatus 100 is explained concretely as follows. FIG. 4 is anexplanatory block diagram showing the picture classification processunit 5 whereas FIGS. 5 and 6 are diagrams each showing the principle ofoperation of the picture classification processing carried out by thevideo classification process unit 5.

In this embodiment, the video classification process unit 5 detectscharacteristics of the video data from the video data compressed bytypically adopting the MPEG2 method in order to classify pictures.

When a characteristic-data detection unit 51 employed in the videoclassification process unit 5 receives an I picture from the videoencoder 4, the characteristic-data detection unit 51 divides the framedata of the I picture into 25 picture areas or 25 divided-picture areaslike those shown in FIG. 5. Then, the characteristic-data detection unit51 generates a luminance vector Ay, which is a 25^(th)-degree vectorhaving 25 components as expressed by Eq. (1) shown in FIG. 6. The 25components are average values of DC coefficients selected among DCT(Discrete Cosine Transform) components of Y components (or luminancecomponents) in the 25 picture areas respectively. By the same token, thecharacteristic-data detection unit 51 generates a blue color-differencevector Ab, which is a 25^(th)-degree vector having 25 components asexpressed by Eq. (2) shown in FIG. 6. The 25 components are averagevalues of DC coefficients selected among DCT components of Cb components(or blue color-difference components) in the 25 picture areasrespectively. In the same way, the characteristic-data detection unit 51generates a red color-difference vector Ar, which is a 25^(th)-degreevector having 25 components as expressed by Eq. (3) shown in FIG. 6. The25 components are average values of DC coefficients selected among DCTcomponents of Cr components (or red color-difference components) in the25 picture areas respectively.

The characteristic-data detection unit 51 supplies the pieces of vectordata generated sequentially in this way to a data buffer 52, in whichthe data stored, and to a processing unit 53. The data buffer 52 and theprocessing unit 53 are included in the configuration shown in FIG. 4.The processing unit 53 computes an inter-vector distance Vd betweenreference vector data at a classification-processing time and the vectordata of each frame in accordance with Eq. (4) shown in FIG. 6. Thevector data of each frame has already been stored in the data buffer 52for all frames. The reference vector data at theclassification-processing time is vector data most recently supplied bythe characteristic-data detection unit 51 to the processing unit 53.Then, the processing unit 53 finds the shortest distance Vdmin among theinter-vector distances Vd computed for all the frames.

A ID buffer 55 is a memory used for storing IDs (identifications)assigned to the pieces of vector data, which have been stored in thedata buffer 52. The IDs are assigned to the pieces of vector data on aone-to-one basis. That is to say, the IDs are assigned to the frames,whose pieces of vector data have bee stored in the data buffer 52, on aone-to-one basis.

Then, on the basis of processing results output by the processing unit53, the system controller 17 refers to the ID buffer 55 to obtain an IDassigned to a specific frame with a vector having an inter-vectordistance from the reference vector equal to the shortest distance Vdmin.Subsequently, if the shortest distance Vdmin does not exceed a thresholdvalue Th determined in advance, the system controller 17 determines thespecific frame with a vector having an inter-vector distance from thereference vector equal to the shortest distance Vdmin to be a framesimilar to the frame having the reference vector. In this case, anidentification IDdet, which is the ID of the specific frame, is outputas an identification information signal, bypasses an ID generation unit54, and is stored in the ID buffer 55. As implicitly explained in adescription given earlier, the specific frame and the frame having thereference vector are each an I picture.

The ID generation unit 54 included in the configuration shown in FIG. 4is a 16-bit counter having an initial value of 0 (zero). If the shortestdistance Vdmin exceeds the threshold value Th determined in advance, thesystem controller 17 determines the specific frame (an I picture) with avector having an inter-vector distance from the reference vector equalto the shortest distance Vdmin to be a frame not similar to the frame(an I picture) having the reference vector. In this case, the currentvalue of the ID generation unit 54 is output as an ID assigned to theframe having the reference vector and also stored in the ID buffer 55.Then, the ID generation unit 54 increments the contents thereof by 1 togenerate an ID to be assigned to a next processed frame.

As described above, the same ID is assigned to frames (I pictures)having a high degree of similarity. Thus, a segment consisting of frameshaving the same ID can be determined to be a similar-picture segment. Inaddition, similar-picture segments separated from each other by othersegments can be determined to be segments pertaining to a group ofsimilar-picture segments resembling each other.

Moreover, the video classification process unit 5 is capable ofdetermining whether or not a specific I picture completing theclassification process is a picture located at a scene-change point (ora cut point) from a difference between the I picture completing theclassification process and an I picture immediately preceding thespecific I picture. By detecting scene-change points in this way, it ispossible to easily implement the process to generate shrunk pictures foronly original pictures located at the scene-change points and displaythe shrunk pictures on the classification display unit 19 as thumb-nailpictures as shown in FIG. 2C and the process to multiplex the shrunkpictures into the aforementioned multiplexed data to be stored onto therecording medium.

Then, the system controller 17 of this embodiment creates anidentification information signal on the basis of information receivedfrom the video classification process unit 5 and supplies this signal tothe multiplexing process unit 6. The identification information signalincludes information identifying a recorded television broadcastprogram, information identifying each shrunk picture of the televisionbroadcast program, information indicating the position of eachidentified shrunk picture on a multi-picture display and informationidentifying an original picture, for which the identified shrunk pictureis generated.

It is to be noted that the information indicating the position of anidentified shrunk picture on a multi-picture display can be found fromthe number of I pictures of a specific similar-picture segment and agroup of similar-picture segments. As the information identifying anoriginal picture, for which the identified shrunk picture is generated,a frame number or a timestamp can be used.

<Sound Classification Processing>

The following description concretely explains processing carried out bythe audio classification process unit 20 employed in therecording/reproduction apparatus 100 described above to classify sounds.FIG. 7 is an explanatory block diagram showing the concreteconfiguration of the sound classification process unit 20. The audioclassification process unit 20 of this embodiment can be considered tobe a unit including a musical-signal portion for recognizing a musicalaudio signal and a voice-signal portion for recognizing aconversational-voice audio signal.

A digital audio signal generated by the audio A/D converter 1 issupplied to a predetermined signal segment detection unit 201 employedin the audio classification process unit 20. The predetermined signalsegment detection unit 201 extracts an audio-signal segment with alength in the range 0.5 to 5 seconds from the digital audio signalsupplied thereto and supplies the extracted audio-signal segment to anFFT signal process unit 202.

The audio-signal segment extracted from the digital audio signal has alength in a predetermined range as described above because, if theduration of the audio-signal segment is too short, it is impossible todetermine whether the audio signal is a musical audio signal or aconversational-voice audio signal with a high degree of accuracy.

The FFT signal process unit 202 carries out FFT (Fast FourierTransformation) processing on the digital audio signal supplied theretoto transform the signal into information on frequency components. Then,the FFT signal process unit 202 supplies the information on frequencycomponents to a peak-frequency detection unit 203. The peak-frequencydetection unit 203 detects spectrum-peak frequencies from theinformation on frequency components of the supplied digital audio signaland supplies the detected spectrum-peak frequencies to thepeak-frequency continuity determination unit 204.

On the basis of the supplied spectrum-peak frequencies, thepeak-frequency continuity determination unit 204 determines whether ornot the spectrum-peak frequencies exist continuously in a predeterminedrange in order to determine whether or not the audio signal beingprocessed is a musical audio signal.

The peak-frequency continuity determination unit 204 supplies a resultof the determination to the multiplexing process unit 6 as explainedearlier by referring to FIG. 1. The multiplexing process unit 6multiplexes the result of the determination into the multiplexed signalas information indicating whether or not the audio signal beingprocessed is a musical audio signal. The peak-frequency continuitydetermination unit 204 also supplies the result of the determination tothe classification display unit 19 by way of the system controller 17 ordirectly. The classification display unit 19 uses the result of thedetermination in a predetermined screen to display thumb-nail picturesof the audio signal by classifying the thumb-nail pictures into amusical portion and a non-musical portion.

In addition, the determination result output by the audio classificationprocess unit 20 can also be used to classify the audio signal into amusical-signal segment and a non-musical-signal segment, and a scene ata point of time marking a change of the segment category of the audiosignal is displayed as a shrunk picture in the same way as the displayof a thumb-nail picture located at a scene-change point describedearlier. In this case, the broadcast program is displayed by clearlyseparating the musical and explanatory portions of the program from eachother so that it is possible to reproduce for example only the musicalportion or the explanatory portion.

<Reproduction System>

Next, the reproduction system is explained. A control signal generatedby the system controller 17 controls the reproduction control unit 9 toread out multiplexed data specified by the user from the recordingmedium 8 and supply the data to the reproduction process unit 10. Thereproduction process unit 10 carries out predetermined processesincluding deinterleave processing on the multiplexed data suppliedthereto and supplies the processed multiplexed data to the separationprocess unit 11. The separation process unit 11 demultiplexes themultiplexed data supplied thereto to generate audio data, video data,shrunk-picture data and an identification information signal to mentiona few.

The audio data generated by the separation process unit 11 is suppliedto an audio decoder 12. The audio data supplied to the audio decoder 12is data compressed by adoption of a predetermined compression method.The audio decoder 12 thus decompresses the compressed audio datasupplied thereto and supplies audio data obtained as a result of thedecompression to an audio digital/analog converter 13, which isabbreviated hereafter to an audio D/A converter.

The audio D/A converter 13 converts the digital audio signal suppliedthereto into an analog audio signal and outputs the analog audio signalto a speaker. The speaker generates a sound according to the analogaudio signal supplied thereto.

In addition, the video data generated by the separation process unit 11is supplied to a video decoder 14. The video data supplied to the videodecoder 14 is data compressed by adoption of a predetermined compressionmethod. The video decoder 14 thus decompresses the compressed video datasupplied thereto and supplies video data obtained as a result of thedecompression to a video digital/analog converter 15, which isabbreviated hereafter to a video D/A converter.

The video D/A converter 15 converts the digital video signal suppliedthereto into an analog video signal and outputs the analog video signalto a monitor receiver. The monitor receiver displays a picture accordingto the analog video signal supplied thereto.

The identification information signal generated by the separationprocess unit 11 is supplied to an identification-signal detection unit16. The identification-signal detection unit 16 detects necessaryidentification information signal from the identification informationsignal supplied thereto and supplies the identification information tothe system controller 17 and the classification display unit 19.

In addition, the identification-signal detection unit 16 also extracts apredetermined timestamp signal or frame numbers from the identificationinformation signal supplied thereto and supplies the timestamp signal orthe frame numbers to the system controller 17. By detecting areproduction point of time on the basis of the timestamp signal, areproduction position can also be displayed on the classificationdisplay unit 19.

Then, when a command is entered via the user input I/F 18 as a commandmaking a request for a display classifying a broadcast program recordedon the recording medium 8, the system controller 17 controls thereproduction control unit 9 on the basis of the identificationinformation signal supplied to the system controller 17 to read outshrunk pictures of the desired broadcast program from the recordingmedium 8 and supplies the shrunk pictures to the classification displayunit 19 by way of the reproduction process unit 10 and the separationprocess unit 11. In addition, the system controller 17 controls theclassification display unit 19 on the basis of the identificationinformation signal to display a classification of the broadcast programrecorded on the recording medium 8 as shown in FIG. 3.

As described above, even with a predetermined timing after the end ofthe recording process, by using the identification information signal aswell as the shrunk-picture signal, which have been generated on thebasis of the picture-similarity determination processing and theclassification processing and then stored in the recording medium, aclassification of pictures can be displayed as shown in FIG. 3.

It is to be noted that this embodiment has been exemplified by giving acase of recording one television broadcast program as an example.However, the embodiment is by no means limited to such a case. Forexample, the embodiment can also be applied to display a classificationobtained as a result of picture-similarity determination processing andclassification processing, which are carried out for a specifiedchannel, a specified program segment or a specified time segment.Typically, the specified channel, the specified program segment or thespecified time segment may involve all programs broadcasted through apredetermined channel, a portion of a predetermined program or apredetermined time segment of a predetermined broadcasting channel. Thatis to say, the picture-similarity determination processing and theclassification processing are carried out only during a proper segmentto display a classification of pictures for the segment.

In addition, it is also possible to generate picture data for a digestreproduction process and picture data for a skimming reproductionprocess during a recording process and store these pieces of data alongwith the main picture data of the broadcast program so that the digestand skimming reproduction processes of the broadcast program can becarried out.

It is to be noted that, in this specification, the digest reproductionprocess means a process to reproduce a broadcast program recorded on arecording medium by jumping from a location to another in the program soas to allow the user to grasp an outline of the program. For example,the digest reproduction process can be implemented by continuouslyreproducing portions of a broadcast program only from physicallyseparated signal segments, which each precede and succeed a scene changeas well as have an audio level equal to or higher than a predeterminedvalue.

On the other hand, the skimming reproduction process is a process toreproduce only a scene selected among those of a broadcast programrecorded on a recording medium as a scene similar to one specified bythe user.

If it is desired to carry out a predetermined special reproductionprocess such as the digest or skimming reproduction process, the userneeds to enter a command via the user input I/F 18. Receiving thecommand information entered by the user, the system controller 17controls the reproduction control unit 9 to read out picture data forthe desired special reproduction method so as to allow pictures of thedesired special reproduction method to be reproduced.

<Operations of the Recording/Reproduction Apparatus 100>

Next, operations carried out by the recording/reproduction apparatus 100implemented the first embodiment described above in a recording processare explained by referring to flowcharts shown in FIGS. 8 to 11. FIGS. 8and 9 show a flowchart referred to in explanation of operations carriedout by the recording/reproduction apparatus 100 implemented by the firstembodiment. The following description explains a typical case in whichpictures each located at a scene-change point are stored in therecording medium as shrunk pictures as shown in FIG. 2C and the shrunkpictures are displayed as a result of classification at the end of therecording process.

Receiving an operation command entered by the user via the user inputI/F 18, the system controller 17 carries out operations represented bythe flowchart shown in FIGS. 8 and 9. The flowchart begins with a stepS1 to determine whether the process requested by the command is areproduction or recording process. If the determination result obtainedat the step S1 indicates that the requested process is a recordingprocess, the flow of the operations goes on to a step S2 at which thesystem controller 17 controls components employed in therecording/reproduction apparatus 100 to put themselves in a recordingmode starting the process to record audio and video signals onto therecording medium 8.

Then, at the next step S3, the video classification process unit 5detects picture characteristics of each I picture as explained earlierby referring to FIGS. 4 to 6. Subsequently, at the next step S4, thevideo classification process unit 5 carries out the similaritydetermination processing and the classification processing on the Ipictures. The pieces of processing carried out at the step S4 correspondto the processing to generate an identification information signal asdescribed earlier.

Then, at the next step S5, the system controller 17 compares a specificI picture currently being processed with an I picture immediatelypreceding the specific I picture on the basis of a result of theclassification processing in order to determine whether or not thespecific I picture is an I picture located at a cut point (or ascene-change point) by finding out whether or not the specific I pictureis not similar to the immediately preceding I picture. The operation ofthe step S5 is carried out for all the successive I pictures.

Subsequently, at the next step S6, the system controller 17 examines aparticular I picture currently being processed to determine whether ornot the particular I picture is an I picture located at any one of thecut points detected at the step S5. If the particular I picturecurrently being processed is an I picture located at any one of the cutpoints, the flow of the operations goes on to a step S7 at which thesystem controller 17 controls the shrunk-picture process unit 22 togenerate a shrunk picture of the particular I picture and supplies thegenerated shrunk picture to the multiplexing process unit 6.

After the operation carried out at the step S7 is completed or thedetermination result obtained at the step S6 indicates that theparticular I picture currently being processed is not an I picturelocated at any one of the cut points, the flow of the operations goes onto a step S8 at which the system controller 17 creates informationindicating, among others, how many consecutive frames are similar toeach other as information indicating positions, and supplies theinformation indicating positions to the multiplexing process unit 6 at astep 8. It is to be noted that, in the case of MPEG data, timestampinformation can be used as information indicating a position.

Then, at the next step S9, the multiplexing process unit 6 multiplexesthe identification information signal with the original signal to berecorded in order to generate a multiplexed signal and records themultiplexed signal onto the recording medium 8 by way of the recordingprocess unit 7. Comprising audio and video data, the original signal tobe recorded has been supplied to the multiplexing process unit 6earlier. The identification information signal includes the informationindicating positions or a timestamp signal.

Subsequently, at the next step S10, the system controller 17 determineswhether or not the user has entered a command to stop the recordingprocess by way of the user input I/F 18. If the determination resultindicates that the user has not entered a command to stop the recordingprocess, the flow of the operations goes back to the step S3 to repeatthe operations described above from this step in order to continue theprocess to record audio and video signals supplied to therecording/reproduction apparatus 100 onto the recording medium.

If the determination result obtained at the step S10 indicates that theuser has entered a command to stop the recording process, on the otherhand, the flow of the operations goes on to a step S11 to determinewhether or not a classification display has been requested. Typically,prior to the start of the recording process, the user indicates whetherof not a request for a classification display is to be made at the endof the process.

If the determination result obtained at the step S11 indicates that arequest for a classification display is not made, the execution of theoperations represented by the flowchart shown in FIGS. 8 and 9 is ended.Then, the recording/reproduction apparatus 100 typically enters a stateof waiting for the user to enter a next command. If the determinationresult obtained at the step S11 indicates that a request for aclassification display is made, on the other hand, the flow of theoperations goes on to a step S12 at which a classification display isoutput to the classification display unit 19 to show shrunk picturesstored on the recording medium as shown in FIG. 3.

After a picture classification is displayed, the flow of the operationsgoes on to a step S13 at which the system controller 17 determineswhether or not the user has entered a reproduction command. If thedetermination result obtained at the step S13 indicates that the userdid not enter a reproduction command, the execution of the operationsrepresented by the flowchart shown in FIGS. 8 and 9 is ended. Then, therecording/reproduction apparatus 100 typically enters a state of waitingfor the user to enter a next command.

It is to be noted that the recording/reproduction apparatus 100implemented by this embodiment is capable of reproducing only a group ofsimilar-pictures segments, which resemble each other and include anoriginal picture corresponding to a shrunk picture selected from thoseshown in the classification display.

If the determination result obtained at the step S1 indicates that therequested process is a reproduction process or if the determinationresult obtained at the step S13 indicates that the user has entered areproduction command, on the other hand, the flow of the operations goeson to a step S14 of a continuation flowchart shown in FIG. 9. At thisstep, the system controller 17 controls components employed in thereproduction system to put the recording/reproduction apparatus 100 in areproduction mode.

Then, at the next step S15, the system controller 17 determines whetherthe requested reproduction process is a special reproduction process oran ordinary reproduction process. Examples of the special reproductionprocess are the digest reproduction and the skimming reproductionprocesses.

If the determination result obtained at the step S15 indicates that therequested reproduction process is a special reproduction process, theflow of the operations goes on to a step S16 at which the systemcontroller 17 controls components employed in the reproduction system toput the recording/reproduction apparatus 100 in the mode of therequested special reproduction process and carry out the requestedspecial reproduction process. In this requested special reproductionprocess, the digest reproduction, the skimming reproduction or theclassification display reproduction is carried out as a specialreproduction operation requested by the user.

Then, at the next step S17, the system controller 17 determines whetheror not the user has entered a command to stop the special reproductionprocess. If the determination result indicates that the user has entereda command to stop the special reproduction process, the execution of theoperations represented by the flowchart shown in FIGS. 8 and 9 is ended.Then, the recording/reproduction apparatus 100 typically enters a stateof waiting for the user to enter a next command.

If the determination result obtained at the step S17 indicates that theuser has not entered a command to stop the special reproduction process,on the other hand, the flow of the operations goes on to a step S18 todetermine whether or not a recording process has been requested, thatis, whether or not a command has been entered to change the reproductionmode to a recording mode.

If the determination result obtained at the step S18 indicates that arecording process has not been requested, the flow of the operationsgoes back to the step S18 to continue the special reproduction process.If the determination result obtained at the step S18 indicates that arecording process has been requested, on the other hand, the flow of theoperations goes back to the step S1 of the main flowchart shown in FIG.8 to repeat the operations described above all over again from thisstep.

If the determination result obtained at the step S15 indicates that therequested reproduction process is not a special reproduction process, onthe other hand, the flow of the operations goes on to a step S19 atwhich the system controller 17 controls components employed in thereproduction system to put the recording/reproduction apparatus 100 inthe mode of the ordinary reproduction process and carry out the ordinaryreproduction process.

Then, at the next step S20, the system controller 17 determines whetheror not the user has entered a command to stop the ordinary reproductionprocess. If the determination result indicates that the user has entereda command to stop the ordinary reproduction process, the execution ofthe operations represented by the flowchart shown in FIGS. 8 and 9 isended. Then, the recording/reproduction apparatus 100 typically enters astate of waiting for the user to enter a next command.

If the determination result obtained at the step S20 indicates that theuser has not entered a command to stop the ordinary reproductionprocess, on the other hand, the flow of the operations goes on to a stepS21 to determine whether or not a recording process has been requested,that is, whether or not a command has been entered to change thereproduction mode to a recording mode.

If the determination result obtained at the step S21 indicates that arecording process has not been requested, the flow of the operationsgoes back to the step S19 to continue the ordinary reproduction process.If the determination result obtained at the step S21 indicates that arecording process has been requested, on the other hand, the flow of theoperations goes back to the step S1 of the main flowchart shown in FIG.8 to repeat the operations described above all over again from thisstep.

It is to be noted that FIGS. 10 and 11 show a flowchart referred to inexplanation of other operations carried out by therecording/reproduction apparatus to display shrunk pictures of alloriginal pictures in the same frame order as the original pictures asshown in FIG. 2B. The flowchart shown in FIGS. 10 and 11 is all but thesame as the flowchart shown in FIGS. 8 and 9 but, since the shrunkpictures are recorded in the same frame order as the original pictures,the information indicating positions is not required. Thus, theoperation carried out at the step S8 of the flowchart shown in FIG. 8 isnot required in the flowchart shown in FIG. 10.

The other steps of the flowchart shown in FIGS. 10 and 11 are the sameas those of the flowchart shown in FIGS. 8 and 9. In the flowchart shownin FIGS. 10 and 11, the other steps identical with their counterparts inthe flowchart shown in FIGS. 8 and 9 are denoted by the same referencenumerals as the counterparts and the explanations of the counterpartsteps are not repeated.

<Summary of the First Embodiment>

As described above, in the process to record video and audio signals ofa television broadcast program onto a recording medium, therecording/reproduction apparatus 100 implemented by the first embodimentrecognizes characteristics of pictures included in the video signal andcarries out similarity determination processing and classificationprocessing to classify similar pictures on the basis of the recognizedcharacteristics.

At the same time, scene-change points are detected. If a plurality ofsuccessive frames has similar pictures, shrunk pictures of only originalpictures located at the scene-change of the classified pictures arecreated and information indicating positions of the shrunk and originalpictures on the time base or information indicating positions of theframes or the like is detected. Then, an identification informationsignal such as the data of the shrunk pictures and the informationindicating positions is recorded onto the recording medium along withthe original video data.

The shrunk pictures are displayed fast for each classification of theclassification processing included in the recording process at the sametime as the recording process, at the end of the recording process orwith a predetermined timing after the recording process. That is to say,a multi-picture display like the one shown in FIG. 3 appears on thedisplay unit as a classification display. From this classificationdisplay, recorded data can be grasped efficiently and effectively. Inaddition, a special process such as the digest reproduction process canbe carried out immediately.

Thus, this embodiment is capable of solving problems such as inabilityto recognize recorded video data without reproducing the entire videodata as is the case with the conventional VTR (Video Tape Recorder).That is to say, recorded video data can be grasped quickly andaccurately at the same time as the recording process, right after theend of the recording process or with a predetermined timing after therecording process.

It is to be noted that, in the description of the first embodiment, afirst information signal serving as the principal object of recordingincludes video and audio signals of a television broadcast program orthe like. However, the first information signal serving as the principalobject of recording is by no means limited to such signals. For example,the first information signal serving as the principal object ofrecording can be either the video signal or the audio signal only.

If the first information signal is an audio signal only, as a secondinformation signal, it is possible to use typically a display messagecreated in the recording/reproduction apparatus. Examples of such adisplay message are a first musical portion, a second musical portion, amale-voice portion and a female-voice portion.

It is to be noted that, while a musical-sound portion and aconversational-voice portion are detected as described above, amale-voice portion and female-voice portion of an audio signal can bedetected on the basis of frequency components of the audio signal.

In addition, the first information signal can be combination datacomprising a video signal, program data and text data or combinationdata including an audio signal, program data and text data. That is tosay, the present invention can be applied to other cases as long as ineach of the other cases, the similarity determination processing iscarried out in predetermined units and an information signal that can beclassified into similar-picture segments is used as the firstinformation signal.

In addition, the similarity determination processing and theclassification processing can not only be carried out in program units,but also be carried out all the time during a recording process. Asanother alternative, the time base is divided into time segments and thesimilarity determination processing as well as the classificationprocessing can also be carried out for each of the time segments. As afurther alternative, the similarity determination processing and theclassification processing can also be carried out for only a proper unitsuch as a time unit within a program or a first predetermined timeperiod.

In addition, it may be impossible to accommodate a classificationdisplay in a screen. In this case, the classification display can bescrolled in the horizontal (width) direction or the vertical (height)direction so as to make the user capable of viewing all shrunk pictures.

Second Embodiment

FIG. 12 is an explanatory block diagram showing a recording/reproductionapparatus 200 implemented by a second embodiment to which an embodimentof the information-signal process apparatus and information-signalprocess method provided by the present invention is applied. FIGS. 13A,13B, 14 and 15 are explanatory diagrams showing the principle ofoperation of the recording/reproduction apparatus implemented by thesecond embodiment.

The recording/reproduction apparatus 200 implemented by the secondembodiment shown in FIG. 12 has all but the same configuration as thatof the recording/reproduction apparatus implemented by the firstembodiment shown in FIG. 1 except that the recording/reproductionapparatus 200 implemented by the second embodiment does not include theI-picture detection unit 21 and shrunk-picture process unit 22 employedin the recording/reproduction apparatus 100 of the first embodimentshown in FIG. 1 and that the recording/reproduction apparatus 200 isnewly provided with a residual recording capacity detection unit 31, arecording-medium initialized-state detection unit 32 and a displaysystem 33.

For the reason described above, in the recording/reproduction apparatus200 implemented by the second embodiment shown in FIG. 12, componentsidentical with their counterparts employed in the recording/reproductionapparatus 100 implemented by the first embodiment shown in FIG. 1 aredenoted by the same reference numerals as the counterparts and thedetailed explanation of the identical counterparts is not repeated.

In addition to principal information (a first information signal)including all video and audio signals of a television broadcast program,the recording/reproduction apparatus 200 implemented by the secondembodiment stores a digest-use signal (summary information or a secondinformation signal) on the recording medium 8 and is thus capable ofcarrying out a digest reproduction process. The digest-use signalincludes a digest-reproduction video signal (summary video information)and a digest-reproduction audio signal (summary audio information),which are used for carrying out a digest reproduction process on thetelevision broadcast program.

The summary information forms a pair in conjunction with the principalsignal of the television broadcast program. By viewing the summaryinformation reproduced from the recording medium 8, the user can graspan outline of contents of the principal information and determinewhether or not it is necessary to normally reproduce and watch theprincipal information. Thus, since the user can grasp an outline ofcontents of the principal information by merely viewing the summaryinformation reproduced from the recording medium 8, there are many casesin which it is not necessary to normally reproduce the principalinformation.

In addition, even though the storage capacity of the recording medium islarge, the capacity is by no means unlimited. Thus, if the principalinformation and the summary information are always managed as a pair, itis quite within the bounds of possibility that necessary summaryinformation is also inadvertently erased. If a recording area forstoring the principal information and a recording area for storing thesummary information are managed by completely separating them from eachother, however, it is difficult to associate the principal informationwith the summary information.

Thus, in order to solve this problem, in the recording/reproductionapparatus implemented by the second embodiment, principal informationand summary information are managed by associating the principalinformation and the summary information with each other and, when thestorage capacity of a recording area remaining in the recording medium 8becomes insufficient for storing new principal information, principalinformation of a least recent date is erased from the recording medium 8but summary information is deliberately left in the recording medium 8,taking precedence of principal information so that the recording medium8 can be utilized effectively. As a result, an information signal suchas a video signal stored on the recording medium 8 can be usedeffectively and with a high degree of efficiency.

It is to be noted that the summary information is information used forcarrying out a digest reproduction process. An example of the summaryinformation is information created by extracting video and audio signalsof segments, which each precede and succeed a scene change resultingtypically from a change of a camera work as well as have an audio levelequal to or higher than a predetermined value, from the principalsignal.

To put it concretely, summary information like one shown in FIG. 13B iscreated by extracting video and audio signals of segments A1, A2 and A3,which each precede and succeed a scene change as shown in FIG. 13A aswell as have an audio level equal to or higher than a predeterminedvalue.

In addition, as another example, summary information for digestreproduction use may also be conceivably created from audio and videosignals of similar portions detected as portions having similar audiosignals of a television broadcast program. Consider a news program asthis other example. In this case, by merely reproducing a scene of theappearance of a newscaster, it is conceivably possible to grasp anoutline of the news. In addition, in the case of a popular-song program,by merely carrying out a digest reproduction process on a scenebroadcasting music, the contents of the program can be grasped.

It is to be noted that, in the second embodiment described above,summary information is created by extracting video and audio signals ofsegments, which each precede and succeed a scene change and have anaudio level equal to or higher than a predetermined value, as shown inFIGS. 13A and 13B.

<Principle of Operation>

First of all, the principle of operation of the recording/reproductionapparatus 200 implemented by the second embodiment is explained. As alsodescribed earlier, the recording/reproduction apparatus 200 implementedby the second embodiment is capable of storing both principalinformation, which includes all video and audio signals of a televisionbroadcast program, and summary information on the recording medium 8.The summary information is the so-called abstract of the principalinformation. The summary information is a digest-reproduction signalused to grasp an outline of a television broadcast program, which canotherwise be viewed only by reproducing the principal information.

In addition, in the case of the recording/reproduction apparatus 200implemented by the second embodiment, the recording area of therecording medium 8 is divided into an area R1 used as a recording areafor storing the summary information only and an area R2 used as arecording area for storing the principal information only as shown inFIG. 14A. Thus, the summary information is stored in the recording areaR1 of the recording medium 8 while the principal information is storedin the recording area R2 of the recording medium 8.

The system controller 17 employed in the recording/reproductionapparatus 200 monitors a residual storage capacity of the recordingmedium 8 through the residual recording capacity detection unit 31. Thefollowing description explains a case in which the remaining storagecapacity of the recording area R2 for storing principal informationbecomes insufficient as detected in an operation to record the summaryinformation and the principal information, which pertain to a fifthprogram, after operations to store the summary information and theprincipal information are completed for first to fourth programs asshown in FIG. 14A.

In such a case, the summary information of the fifth program is storedas it is in the recording area R1 provided in the recording medium 8 asan area allocated to summary information, and the principal informationof the fifth program is stored in a recording area for storing theprincipal information of the first program recorded on a least recentrecording date, overwriting (being superposed on) the principalinformation of the first program as shown in FIG. 14B.

If the principal information of the fifth program cannot all beaccommodated in the recording region for storing the principalinformation of the first program, the remaining information of theprincipal information of the fifth program is recorded in a regionfollowing a recording region for storing the principal information ofthe fourth program.

By storing the information of the fifth program as described above,however, even though the principal information of the first program isunavoidably erased, the principal information of the fifth program,which is the most recent program, can be stored in the recording medium8. On top of that, since the summary information of the first program isnot deleted, by carrying out a digest reproduction process to reproducethe summary information of the first program at any time, an outline ofthe first program can be grasped.

It is to be noted that, if it is feared that the size of the recordingarea R1 itself left in the recording medium 8 also becomes insufficient,the system controller 17 reports this situation to the user through thedisplay system 33, to prompt the user to pay attention to thissituation. That is to say, the user is requested to rearrangeinformation already stored on the recording medium 8 or extend therecording area thereof.

In the case of the example shown in FIG. 14, the recording area of therecording medium 8 is divided into the recording area R1 allocated tosummary information and the recording area R2 allocated to principalinformation in advance as described above. It is worth noting, however,that the use of the recording area of the recording medium 8 is by nomeans limited to such division. For example, each principal informationand summary information for the principal information can also be storedin areas adjacent to each other as shown in FIG. 15A. As the remainingrecording area is expected to become insufficient, the principalinformation of the fifth program, which is the most recent program, isstored in a recording area for storing the principal information of thefirst program recorded on a least recent recording date, overwriting(being superposed on) the principal information of the first program asshown in FIG. 15B.

In a word, by replacing the least recent principal information by mostrecent principal information but leaving the summary information for theleast recent principal information in the recording area as it is, it ispossible to avoid a situation in which summary information associatedwith principal information of interest to the user is also inevitablydeleted so that an outline of the principal information of interest tothe user cannot be grasped.

<Operations and Configuration of the Recording/Reproduction Apparatus200 Implemented by the Second Embodiment>

The following description explains the configuration of therecording/reproduction apparatus 200 implemented by the secondembodiment shown in FIG. 12 and operations carried out thereby. As shownin FIG. 12, the recording/reproduction apparatus 200 implemented by thesecond embodiment includes the recording-medium initialized-statedetection unit 32. The recording-medium initialized-state detection unit32 is a component for determining whether or not the recording medium 8is an initialized state, which is a state of no information such asvideo data and audio data recorded on the recording medium 8.

A determination result produced by the recording-mediuminitialized-state detection unit 32 is reported to the system controller17. If the result of determination indicates the initialized state of noinformation signal recorded on the recording medium 8, the systemcontroller 17 reports this state to the user through the display system33 to prompt the user to enter an input of selecting either the firstrecoding method shown in FIG. 14 or the second recoding method shown inFIG. 15 as a technique for recording information signals onto therecording medium 8.

Assume for example that the user prefers the first recoding method shownin FIG. 14. In this case, information signals can be managed by dividingthe recording medium 8 into the recording area R1 allocated to summaryinformation and the recording area R2 allocated to principalinformation. Thus, the recording medium 8 is convenient for a case inwhich pieces of principal information recorded for typically severalweeks need to be accommodated but pieces of summary information recordedfor at least several months must be saved.

In the case of the first recoding method, no warning is issued till therecording area allocated to summary information becomes insufficient.Thus, the first recoding method offers a merit that least past summaryinformation can be stored and kept with a high degree of reliability andmost recent pieces of principal information can be recorded and savedreliably without frequently rearranging information already stored inthe recording medium 8.

In the case of the second recoding method shown in FIG. 15, on the otherhand, it is not necessary to allocate large first and second recordingareas in advance. Thus, the recording area can be used with a highdegree of efficiency. In addition, since there is neither recording areaallocated specially to summary information nor recording area allocatedspecially to principal information, the second recoding method also hasa merit of an ability to record summary and principal information over arelatively long period of time. As the recording area is all used up forstoring information, however, there may be generated a case in whichboth summary information and principal information can no longer berecorded.

Either the first recording method or the second recording method needsto be selected in accordance with the way in which the user wants toutilize the recording/reproduction apparatus. The user enters an inputspecifying which recording method is to be adopted to therecording/reproduction apparatus through the user input I/F 18. Such aninput is then set in typically a non-volatile memory employed in thesystem controller 17. Every time a recording process is carried out, thenon-volatile memory is referred to for the recording method to beadopted so that an information signal can be recorded onto the recordingmedium 8 in accordance with the recording method preferred by the user.

It is to be noted that, typically, the first recoding method shown inFIG. 14 is set in advance in the memory as a default recording method.If necessary, however, the default recording method can be replaced withthe second recoding method shown in FIG. 15. It is needless to say that,conversely, the second recoding method shown in FIG. 15 may be set inadvance in the memory as a default recording method and, if necessary,the default recording method can be replaced with the first recodingmethod shown in FIG. 14.

Then, much like the recording/reproduction apparatus 100 implemented bythe first embodiment described earlier, in the recording/reproductionapparatus 200 implemented by the second embodiment, the analog audiosignal of a television broadcast program is processed by the audio A/Dconverter 1 and the audio encoder 2 to generate compressed audio data,which is then supplied to the multiplexing process unit 6. On the otherhand, the analog video signal of the television broadcast program isprocessed by the video A/D converter 3 and the video encoder 4 togenerate compressed video data, which is then supplied to themultiplexing process unit 6. The multiplexing process unit 6subsequently multiplexes the video and audio data to produce principalinformation.

Much like the video classification process unit 5 employed in therecording/reproduction apparatus 100 implemented by the first embodimentdescribed earlier, the video classification process unit 5 employed inthe recording/reproduction apparatus 200 implemented by the secondembodiment carries out the picture-similarity determination processingto classify the principal information into similar-picture segments andnon-similar-picture segments, supplying results of the determination tothe system controller 17. In this case, the video classification processunit 5 supplies an identification information to the multiplexingprocess unit 6 as a signal conveying required information to be added tothe principal information.

Much like the audio classification process unit 20 employed in therecording/reproduction apparatus 100 implemented by the first embodimentdescribed earlier, the audio classification process unit 20 employed inthe recording/reproduction apparatus 200 implemented by the secondembodiment is capable of recognizing a musical-signal segment and aconversational-voice-signal segment. In the case of the secondembodiment, however, the recording/reproduction apparatus 200 alsodetects the audio level of an audio signal and supplies information onthe detected audio level to the system controller 17. In this case, theaudio classification process unit 20 supplies an identificationinformation to the multiplexing process unit 6 as a signal conveyingrequired information to be added to the principal information.

On the basis of the determination results received from the videoclassification process unit 5 and the information received from theaudio classification process unit 20 as information on a detected audiolevel, the system controller 17 identifies predetermined video-signaland audio-signal segments, which each precede and succeed a picturelocated at a scene-change point as well as have an audio level equal toor higher than a predetermined value, as segments to be used in adigest-reproduction process, and supplies information on the identifiedsegments to the multiplexing process unit 6.

On the basis of the segment identification information received from thesystem controller 17, the multiplexing process unit 6 extractsinformation of the predetermined segments from the principal informationas summary information to be used in a digest reproduction process. Thatis to say, the multiplexing process unit 6 creates the summaryinformation from the principal information in accordance with controlexecuted by the system controller 17.

The principal and summary information created by the multiplexingprocess unit 6 is recorded onto the recording medium 8 by way of therecording process unit 7. In this case, the system controller 17controls the recording process unit 7 to record the principal andsummary information onto the recording medium 8 by adoption of eitherthe first recording method shown in FIG. 14 or the second recordingmethod shown in FIG. 15 as a recording technique selected in advance asdescribed above.

As a result, the principal and summary information is recorded onto therecording medium 8 as shown in FIG. 14 or 15. In the second embodiment,the recording process unit 7 has not only a function to create arecording-use signal but also a recording head unit for recording thecreated recording-use signal onto the recording medium. The systemcontroller 17 controls the position of the recording head unit to selecta recording region into which the recording-use signal is to berecorded.

It is to be noted that, much like the classification display of therecording/reproduction apparatus 100 implemented by the firstembodiment, shrunk pictures of original pictures located at scene-changepoints can be displayed on the classification display unit 19 at arecording time, or pictures and a sound, which are extracted for adigest reproduction process, can also be output to the classificationdisplay unit 19.

In addition, an information signal already recorded on the recordingmedium 8 is reproduced in the same way as the recording/reproductionapparatus 100 implemented by the first embodiment. If the determinationresult indicates that the user enters a command making a request for adigest reproduction process of a program by operating the user input I/F18, however, the system controller 17 controls the reproduction controlunit 9 to read out summary data of a program specified in the commandfrom the recording medium 8 and supply the summary data to theseparation process unit 11 by way of the reproduction process unit 10.The separation process unit 11 separates audio and video portions of thesummary data from each other.

Then, the separation process unit 11 outputs the audio portion of thesummary data to a speaker by way of the audio decoder 12 and the audioD/A converter 13 and outputs the video portion of the summary data to amonitor receiver by way of the video decoder 14 and the video D/Aconverter 15 in the digest reproduction process.

It is to be noted that the video classification process unit 5 and theaudio classification process unit 20 are both capable of carrying outthe classification processing on data prior to the compression process.

<Operations of the Recording/Reproduction Apparatus 200>

Next, operations carried out by the recording/reproduction apparatus 200implemented by the second embodiment are explained by referring to aflowchart shown in FIGS. 16 and 17. In order to make the explanationsimple, the flowchart shown in FIGS. 16 and 17 assumes a typical case inwhich a recording time interval has been set in advance in terms of arecording start time and a recording end time as is the case with timervideo recording or reservation video recording.

When the user requests that the timer recording be started, the systemcontroller 17 computes a recording time interval (from the specifiedrecording start time to the specified recording end time) by subtractingthe specified recording start time from the specified recording end timeat a step S101.

Then, at the next step S102, the system controller 17 computes aresidual storage capacity of the recording area R1 allocated to summaryinformation on the basis of a detection output received from theresidual recording capacity detection unit 31. Subsequently, at the nextstep S103, the system controller 17 determines whether or not thecomputed residual storage capacity is equal to or greater than a valuedetermined in advance, that is, whether or not the computed residualstorage capacity is equal to or greater than the amount of summaryinformation estimated for the recording time interval found at the stepS101.

If the determination result obtained in the operation carried out at thestep S103 indicates that the computed residual storage capacity of therecording area R1 allocated to summary information is equal to orgreater than the value determined in advance, the flow of the operationsgoes on to a step S104 at which the system controller 17 computes aresidual storage capacity of the recording area R2 allocated toprincipal information on the basis of a detection output received fromthe residual recording capacity detection unit 31. Subsequently, at thenext step S105, the system controller 17 determines whether or not thecomputed residual storage capacity is equal to or greater than acapacity determined in advance, that is, whether or not the computedresidual storage capacity is equal to or greater than the amount ofprincipal information estimated for the recording time interval found atthe step S101.

If the determination result obtained in the operation carried out at thestep S105 indicates that the computed residual storage capacity of therecording area R2 allocated to principal information is equal to orgreater than the value determined in advance, the flow of the operationsgoes on to a step S114 at which the system controller 17 allocates arecording area in the recording medium 8 to the current principalinformation to be recorded this time. Then, at the next step S108, thesystem controller 17 records the principal information into theallocated area of the recording medium 8.

If the determination result obtained in the operation carried out at thestep S105 indicates that the computed residual storage capacity of therecording area R2 allocated to principal information is not sufficient,on the other hand, the flow of the operations goes on to a step S106 atwhich the system controller 17 identifies a recording region used forrecording the least recent principal information among pieces ofprincipal information already recorded on the recording medium 8. Then,at the next step S107, the system controller 17 starts controlling therecording process unit 7 to record the current principal informationinto the identified recording region. Subsequently, at the next stepS108, the recording process unit 7 overwrites the current principalinformation being recorded this time on the least recent principalinformation recorded in the identified recording region.

Then, at the next step S109, characteristics of the principalinformation are detected by carrying out the picture similarityprocessing and the picture classification processing in the videoclassification process unit 5 as well as audio-level detectionprocessing in the audio classification process unit 20. Subsequently, atthe next step S110, the system controller 17 identifies signal segmentsto be used in a digest reproduction process, and determines whether theprincipal information being processed is principal information of asignal segment for which summary information is to be created.

If the determination result obtained in the operation carried out at thestep S110 indicates that the principal information being processed isprincipal information of a signal segment for which summary informationis to be created, the flow of the operations goes on to a step S111 atwhich operations such as identification processing are carried out toextract the principal information from the segment to be used as thesummary information and to attach additional information to theextracted summary information to create the summary information. Theadditional information attached to the summary information isinformation that can be used to indicate that the summary information isof the principal information. Then, at the next step S112, the summaryinformation is recorded into the recording area R1 allocated to summaryinformation.

If the determination result obtained in the operation carried out at thestep S110 indicates that the principal information being processed isnot principal information of a signal segment for which summaryinformation is to be created, on the other hand, the flow of theoperations goes on to a step S113. The flow of the operations also goeson to a step S113 after the operation carried out at the step S112 iscompleted. At the step S113, the system controller 17 determines whetheror not the user has entered a command to stop the recording process. Ifthe system controller 17 determines that the user has not entered acommand to stop the recording process, the flow of the operations goesback to the step S108 to repeat the operations described above from thisstep.

If the determination result obtained in the operation carried out at thestep S113 indicates that the user has entered a command to stop therecording process, on the other hand, the execution of the operationsrepresented by the flowchart shown in FIGS. 16 and 17 is ended. Then,the recording/reproduction apparatus 200 typically enters a state ofwaiting for the user to enter a next command.

If the determination result obtained in the operation carried out at thestep S103 indicates that the computed residual storage capacity of therecording area R1 allocated to summary information is not sufficient, onthe other hand, the flow of the operations goes on to a step S115 of acontinuation flowchart shown in FIG. 17. At the step S115, the systemcontroller 17 issues a warning to the user by way of the display system33. Then, at the next step S116, the system controller 17 determineswhether or not the user has entered a command to stop the recordingprocess. If the determination result obtained in the operation carriedout at the step S116 indicates that the user has entered a command tostop the recording process, the system controller 17 ends the executionof the operations represented by the flowchart shown in FIGS. 16 and 17.

If the determination result obtained in the operation carried out at thestep S116 indicates that the user did not enter a command to stop therecording process or the user has entered a command to continue therecording process, on the other hand, the flow of the operations goes onto a step S117 at which the system controller 17 identifies the leastrecent summary information among pieces of summary information alreadyrecorded in the recording area R1 allocated to summary information.Then, at the next step S118, the system controller 17 sets the recordingregion used for storing the identified least recent summary informationas a recording region for storing the current summary information.Subsequently, the flow of the operations goes back to the step S104 ofthe main flowchart shown in FIG. 16 to repeat the operations describedabove from this step.

The following description explains operations carried out by therecording/reproduction apparatus 200 for a case in which the recordingmedium 8 is still in an initialized state as it is, allowing a recordingmethod to be selected. FIG. 18 shows a flowchart referred to inexplanation of the other operations carried out by therecording/reproduction apparatus 200 for a case in which the recordingmedium 8 is still in an initialized state as it is, allowing a recordingmethod to be selected.

The operations represented by the flowchart shown in FIG. 18 are aprocess carried out in accordance with a command entered as a requestfor execution of the process such as a setting process including a timerrecording process. The processing represented by the flowchart shown inFIG. 18 begins with a step S131 at which the system controller 17recognizes the state of the recording medium 8 on the basis of adetection result received from the recording-medium initialized-statedetection unit 32.

Then, at the next step S132, the system controller 17 determines whetheror not the recording medium 8 is still in an initialized state as it iswith no information signal recorded thereon.

If the determination result obtained in the operation carried out at thestep S132 indicates that an information signal has been recorded on therecording medium 8, the recording/reproduction apparatus 200 exits fromthe process represented by the flowchart shown in FIG. 18. Then,typically, the recording/reproduction apparatus 200 carries out theprocess represented by the flowchart shown in FIGS. 16 and 17.

If the determination result obtained in the operation carried out at thestep S132 indicates that the recording medium 8 is still in aninitialized state as it is, on the other hand, the flow of the processgoes on to a step S133 at which the system controller 17 shows a messageto the user through the display system 33. The message prompts the userto enter an input of selecting a recording method. The system controller17 then receives an input entered by the user through the user input I/F18 as the input of selecting a recording method.

Then, at the next step S134, the system controller 17 determines whetheror not the input of selecting a recording method selects the first orsecond recording method. If the determination result obtained in theoperation carried out at the step S134 indicates that the input ofselecting a recording method selects the first recording method, theflow of the process goes on to a step S135 at which the systemcontroller 17 sets data, which indicates that the first recording methodhas been selected, in a non-volatile memory employed in the systemcontroller 17.

If the determination result obtained in the operation carried out at thestep S134 indicates that the input of selecting a recording methodselects the second recording method, on the other hand, the flow of theprocess goes on to a step S136 at which the system controller 17 setsdata, which indicates that the first recording method has been selected,in a non-volatile memory employed in the system controller 17. After theoperation carried out at the step S135 or S136 has been completed, therecording/reproduction apparatus 200 exits from the process representedby the flowchart shown in FIG. 18. Then, typically, therecording/reproduction apparatus 200 carries out the process representedby the flowchart shown in FIGS. 16 and 17. It is to be noted that theprocess represented by the flowchart shown in FIG. 18 can be modified tobypass the operation in case a recording method has been selected beforeand the data indicating the selected recording method has already beenset.

<Summary of the Second Embodiment>

As described above, the recording/reproduction apparatus 200 implementedby the second embodiment manages principal information and summaryinformation by associating a recording area allocated to the principalinformation with a recording area allocated to the summary information.If the residual storage capacity of the recording medium 8 including therecording areas becomes insufficient, summary information isdeliberately left in the recording area of the recording medium 8,taking precedence of principal information associated with the summaryinformation so that the recording medium 8 can be used with a highdegree of efficiency. As a result, an information signal such as a videosignal recorded on the recording medium 8 can be utilized effectivelyand with a high degree of efficiency.

That is to say, by preferentially leaving summary information in therecording area of the recording medium 8, it is possible to preventnecessary information from being erased mistakenly and to utilize therecording medium with a high degree of efficiency.

In addition, since summary information is preferentially left in therecording area of the recording medium 8, a digest reproduction processof the summary information can be carried out at any time, making itpossible to grasp an outline of the principal information associatedwith the summary information with a high degree of effectiveness in ashort period of time. As a result, the principal information can bereproduced less frequently and, even if principal information isdeleted, most other pieces of desired principal information can still bereproduced.

On top of that, even in operations to copy and move information alreadyrecorded on a recording medium to another recording medium, the amountof processing to complicatedly process the recorded information can bereduced so that the processing and rearrangement of the recordedinformation can be carried out with a high degree of efficiency.

It is to be noted that, the second embodiment has been explained bytaking video and audio signals of a television broadcast program as apair of information signals to be recorded. However, the scope of thepresent invention is by no means limited to this second embodiment. Thatis to say, the information signal can be only the video signal or onlythe audio signal. As another alternative, the information signal canalso be text data.

In addition, in accordance with the explanation, the recording medium 8is used mainly for recording video and audio signals. However, therecording medium 8 may include a recording area for storing a programand a recording area for storing various kinds of data.

On top of that, in accordance with the recording method shown in FIG.15, summary information and principal information forming a pair arestored in recording regions adjacent to each other as described above.However, it is not always necessary to allocate adjacent recordingregions to such a pair. Of course, it is possible to allocate arecording region to a piece of summary information at a locationseparated away from a recording region to a piece of principalinformation forming a pair with the piece of summary information.

Moreover, it is sufficient to create summary information having such anamount that the summary information can be used to grasp an outline ofinformation represented by the entire principal information associatedwith the summary information. Thus, in the second embodiment, summaryinformation is generated by extracting portions of principal informationfor which the summary information is created. The way to generatesummary information is by no means limited to the technique adopted bythe second embodiment. For example, summary information can also begenerated as information different from principal information for whichthe summary information is created and used in a digest reproductionprocess.

Third Embodiment

FIG. 19 is an explanatory block diagram showing a recording/reproductionapparatus 300 implemented by a third embodiment to which an embodimentof the information-signal process apparatus and information-signalprocess method provided by the present invention is applied.

The recording/reproduction apparatus implemented by the third embodimentshown in FIG. 19 is different from the recording/reproduction apparatus200 implemented by the second embodiment shown in FIG. 12 in that therecording/reproduction apparatus implemented by the third embodimentdoes not have the recording-medium initialized-state detection unit 32employed in the recording/reproduction apparatus 200. As a matter offact, the configuration of the recording/reproduction apparatusimplemented by the third embodiment shown in FIG. 19 is all but the sameas the configuration of the recording/reproduction apparatus 200implemented by the second embodiment shown in FIG. 12 except for thefact that the recording/reproduction apparatus implemented by the thirdembodiment is newly provided with a second recording process unit 41, asecond recording medium 42, a second reproduction control unit 43 and arecording-medium residual-capacity detection unit 44.

In addition, as described above, the recording/reproduction apparatus200 implemented by the second embodiment shown in FIG. 12 includes manyportions each having the same configuration as its counterpart employedin the recording/reproduction apparatus 100 implemented by the firstembodiment shown in FIG. 1. Thus, in the recording/reproductionapparatus 300 implemented by the third embodiment shown in FIG. 19, manyportions having the same configurations as their counterparts employedin the recording/reproduction apparatus 100 implemented by the firstembodiment shown in FIG. 1 or their counterparts employed in therecording/reproduction apparatus 200 implemented by the secondembodiment shown in FIG. 12 are denoted by the same reference numeralsas the counterparts and detailed descriptions of the counterparts arenot repeated.

<Principle of Operation>

Much like the recording/reproduction apparatus 200 implemented by thesecond embodiment, in a process carried out by therecording/reproduction apparatus 300 implemented by the third embodimentto record principal information including all video and audio signalsrepresenting a television broadcast signal onto a recording medium 8,identifications are also recorded on the recording medium 8 asinformation additional to the principal information so that summaryinformation identified by the identifications can be extracted later ina digest-reproduction process to reproduce mutually separated portionsof the principal information as explained earlier by referring to FIGS.13A and 13B.

In the recording/reproduction apparatus 200 implemented by the secondembodiment described earlier, summary information is also created asinformation additional to principal information and recorded on therecording medium. In the case of the recording/reproduction apparatus300 implemented by the third embodiment, after a recording process,summary information is extracted from principal information to be usedin a digest reproduction process.

In the case of the recording/reproduction apparatus 300 implemented bythe third embodiment, however, unlike the recording/reproductionapparatus 200 implemented by the second embodiment described earlier,new principal information is not stored in a region allocated toexisting principal information, overwriting the existing principalinformation even if the storage capacity of the recording medium ispredicted to become insufficient. Instead, the new principal informationis recorded as additional principal information while the existingprincipal information is kept as it is.

Speaking in an easy-to-understand way, the recording/reproductionapparatus 300 has a first recording medium (or a main recording medium)and a second recording medium (or a sub recording medium). Normally,principal information allowing its summary information to be extractedis stored in the first recording medium. When the residual storagecapacity of the first recording medium becomes smaller than apredetermined value, however, principal information already stored inthe first recording medium is moved to the second recording medium inorder to always keep a free area having at least a predetermined size inthe first recording medium. In this way, it is possible to avoid a lackof storage space in the first recording medium. As a result, principalinformation already existing in the first recording medium can beprevented from being deleted due to an operation to overwrite newprincipal information on the existing principal information.

In addition, even if principal information is moved to the secondrecording medium, the summary information associated with the movedprincipal information is left in the first recording medium so that adigest reproduction process can be carried out readily at any time and,if necessary, the principal information moved to the second recordingmedium can also be reproduced in an ordinary reproduction process. Byadoption of such a technique, the good accessibility displayed by thefirst recording medium as a merit of a disk-shaped recording medium isnot lost.

<Operations and Configuration of the Recording/Reproduction Apparatus300 Implemented by the Third Embodiment>

As shown in FIG. 19, the recording/reproduction apparatus 300implemented by the third embodiment includes the second recordingprocess unit 41, the second recording medium 42, the second reproductioncontrol unit 43 and the recording-medium residual-capacity detectionunit 44 in addition to the first recording process unit 7, the firstrecording medium 8, the first reproduction control unit 9 and theresidual recording capacity detection unit 31 respectively.

The first recording medium 8 plays the role of the main recording mediumcited above. In an ordinary recording process, principal and summaryinformation is recorded onto the first recording medium 8. On the otherhand, the second recording medium 42 plays the role of the sub recordingmedium mentioned before. When the residual storage capacity of the firstrecording medium 8 becomes insufficient, principal information alreadystored in the first recording medium 8 is moved to the second recordingmedium 42. Typically, the second recording medium 42 is a diskreproduction device such as a hard disk or a large-capacity recordingmedium such as a magnetic tape.

The recording/reproduction apparatus 300 implemented by the thirdembodiment shown in FIG. 19 is capable of storing an identificationadded to the principal information in a process to record video andaudio signals of a television broadcast program or the like onto thefirst recording medium 8. The identification allows the summaryinformation to be extracted.

Prior to a recording process, for example, the residual recordingcapacity detection unit 31 detects the residual storage capacity of thefirst recording medium 8. If the residual storage capacity is determinedto be insufficient, principal information already stored in the firstrecording medium 8 is moved to the second recording medium 42 by way ofthe second recording process unit 41 to increase the residual storagecapacity of the first recording medium 8. Then, new principalinformation is stored in the recording medium 8.

In an ordinary reproduction process to reproduce principal informationrecorded on the second recording medium 42, data of the principalinformation read out from the second recording medium 42 by the secondreproduction control unit 43 in accordance with control executed by thesystem controller 17 is output to a speaker and a monitor by way of thereproduction process unit 10, the separation process unit 11, the audiodecoder 12, the audio D/A converter 13, the video decoder 14 and thevideo D/A converter 15 in the same way as the process to reproduceprincipal information stored on the recording medium 8.

It is to be noted that the recording-medium residual-capacity detectionunit 44 monitors the residual storage capacity of the second recordingmedium 42 and reports a result of the monitoring to the systemcontroller 17. If the result of the monitoring indicates that theresidual storage capacity of the second recording medium 42 is notadequate, a warning is typically displayed to the user by way of thedisplay system 33 to prompt the user to rearrange information stored inthe second recording medium 42 or replace the second recording medium 42with a new recording medium.

In the case of the recording/reproduction apparatus 300 implemented bythe third embodiment described above, at least, summary information isleft in the first recording medium 8. When the size of the recordingarea in the first recording medium 8 becomes insufficient, principalinformation already stored in the first recording medium 8 is moved tothe second recording medium 42 so that new principal information can bestored onto the first recording medium 8.

It is to be noted that, in the recording/reproduction apparatus 300implemented by the third embodiment shown in FIG. 19, for eachsimilar-picture segment, the video classification process unit 5 and theaudio classification process unit 20 create an ID unique to the segmentand assign the ID to the segment in the same way as therecording/reproduction apparatus 100 implemented by the first embodimentdescribed earlier. In addition, the audio classification process unit 20is also capable of detecting the signal level of an audio signalsupplied thereto.

On the basis of signals output by the video classification process unit5 and the audio classification process unit 20, the system controller 17recognizes signal segments, which each precede and succeed a scenechange as well as have an audio level equal to or higher than apredetermined value as segments of summary information. The controller17 supplies the summary information of the recognized segments to themultiplexing process unit 6 so that the multiplexing unit 6 can addsummary-information identifications to summary-information portions ofthe principal information.

<Operations of the Recording/Reproduction Apparatus 300>

Next, operations carried out by the recording/reproduction apparatus 300implemented by the third embodiment are explained by referring to aflowchart shown in FIGS. 20 to 22. The flowchart shown in FIGS. 20 to 22is a flowchart referred to in explaining recording and reproductionprocesses carried out by the recording/reproduction apparatus 300implemented by the third embodiment. It is to be noted that theprocessing represented by the flowchart shown in FIGS. 20 to 22 iscarried out for each predetermined unit, which can consist of 1 frame toa plurality of frames or 1 GOP to a plurality of GOPs.

The flowchart shown in FIGS. 20 to 22 begins with a step S201 at whichthe system controller 17 determines whether a process specified by anoperation command received from the user by way of the user input I/F 18is a reproduction or recording process. If the determination resultproduced in the operation carried out at the step S201 indicates thatthe requested process is a recording process, the flow of the operationsgoes on to a step S202 at which the system controller 17 detects theresidual storage capacity of the first recording medium 8 through theresidual recording capacity detection unit 31.

Then, at the next step S203, the system controller 17 determines whetheror not the residual capacity is large enough for storing both newprincipal information and new summary information on the basis of thedetection result produced in the operation carried out at the step S202as a result of detecting the residual storage capacity. At the stepS203, for example, a typical threshold value of three hours is used.That is to say, a residual storage capacity corresponding to at leastthree hours is required. It is to be noted that, in the case of a timerrecording mode, for example, a reservation-recording interval can befound from a recording start time and a recording end time. In thiscase, the residual recording capacity is compared with thereservation-recording interval.

If the determination result produced in the operation carried out at thestep S203 indicates that the residual capacity of the first recordingmedium 8 is large enough, the flow of the operations goes on to a stepS204. The system controller 17 controls the audio classification processunit 20 and the video classification process unit 5 to extract signalsof a predetermined segment used as a processing unit from inputinformation (input data) supplied to the audio classification processunit 20 and the video classification process unit 5 respectively. Then,at the next step S205, the audio classification process unit 20 andvideo classification process unit 5 carry out processing to determinesimilar-picture segments on the basis of detected characteristics andclassify the determined similar-picture segments.

Subsequently, at the next step S206, on the basis of results ofprocessing carried out by the audio classification process unit 20 andthe video classification process unit 5, the system controller 17identifies summary-information portions to be used in a digestreproduction process and reports the summary-information portions to themultiplexing unit 6. The multiplexing unit 6 adds identification to amultiplexed signal of the recognized signal segment. Then, at the nextstep S207, the principal information with the identification addedthereto are recorded onto the first recording medium 8 by way of thefirst recording process unit 7.

Also in this third embodiment, a signal segment of principal informationto be used as a segment of summary information is identified on thebasis of results of the picture-similarity determination processing andthe picture classification processing. However, that the presentinvention is by no means limited to the technique adopted by thisembodiment. As described earlier, similar-picture segments can beidentified also for an audio signal. Then, a similar-picture segment anda similar-sound segment can be used as a summary-information segment or,as another alternative, a similar-picture segment is identified by usingonly either video or audio information and a signal segment isrecognized as a segment of summary information.

Then, at the next step S208, the system controller 17 determines whetheror not a command to stop the recording process has been received fromthe user. If the system controller 17 determines that a command to stopthe recording process has been received from the user, the execution ofthe operations represented by the flowchart shown in FIGS. 20 to 22 isended. Then, the recording/reproduction apparatus 300 typically enters astate of waiting for the user to enter a next command. If thedetermination result obtained in the operation carried out at the stepS208 indicates that a command to stop the recording process was notreceived from the user, on the other hand, the flow of the operationsgoes on to a step S209 to determine whether or not the process to recordthe desired broadcast program has been ended from, for example, theadvent of the recording end time or the arrival of another program code.

If the determination result obtained in the operation carried out at thestep S209 indicates that the process to record the desired broadcastprogram has been ended, the execution of the operations represented bythe flowchart shown in FIGS. 20 to 22 is ended. Then, therecording/reproduction apparatus 300 typically enters a state of waitingfor the user to enter a next command. If the determination resultobtained in the operation carried out at the step S209 indicates thatthe process to record the desired broadcast program has not been ended,on the other hand, the flow of the operations goes on to a step S210 todetermine whether or not the user has entered a command to change theoperating mode.

If the determination result produced in the processing carried out atthe step S210 indicates that the user has not entered a command tochange the operating mode, the flow of the operations goes back to thestep S204 to repeat the operations described above from this step tocontinue the process to record a processing unit of principalinformation and summary information to the recording medium 8. If thedetermination result produced in the processing carried out at the stepS210 indicates that the user has entered a command to change theoperating mode, on the other hand, the flow of the operations goes backto the step S201 at which the system controller 17 determines whetherthe process is a reproduction or recording process.

If the determination result produced in the processing carried out atthe step S203 indicates that the residual capacity of the recordingmedium 8 is not sufficient, on the other hand, the flow of theoperations goes on to a step S211 of a continuation flowchart shown inFIG. 21. At the step S211, recording dates of pieces of principalinformation already recorded on the first recording medium 8 areexamined to detect principal information having the oldest date. Then,at the next step S212, a principal signal of a predetermined processingunit of the oldest information is extracted. The principal signal of apredetermined processing unit is also referred to as a principal signalof a predetermined segment. Subsequently, at the next step S213, theprincipal signal of a predetermined segment is copied to the secondrecording medium 42 by way of the second recording process unit 41.

Then, the flow of the operations goes on to a step S214 to detect anidentification added to the copied principal information as anidentification indicating whether or not the segment of the principalinformation is a segment of summary information. Then, at the next stepS215, the detected identification is examined to determine whether ornot the identification indicates that the predetermined segment of theprincipal information copied to the second recording medium 42 is also asummary-information segment usable as summary information.

If the determination result produced by the processing carried out atthe step S215 indicates that the predetermined segment of the principalinformation copied to the second recording medium 42 is also asummary-information segment, the flow of the operations goes on to astep S216. A signal of this portion is moved to a predetermined portionof the first recording medium 8 so as to prevent the pieces of summaryinformation from being scattered at separated locations or to collectthe pieces of summary information pertaining to the broadcast program inthe predetermined portion of the recording medium 8.

If the determination result produced by the processing carried out atthe step S215 indicates that the predetermined segment of the principalinformation copied to the second recording medium 42 is not asummary-information segment, on the other hand, the flow of theoperations goes on to a step S217. Information in the segment is deletedfrom the first recording medium 8 and the area used for storing thedeleted information is used as a free recording area, or the informationis put in status of allowing new data to be overwritten thereupon.

After the processing carried out at the step S216 or S217 has beencompleted, the flow of the operations goes on to a step S218. The systemcontroller 17 controls the audio classification process unit 20 and thevideo classification process unit 5 to extract signals of apredetermined segment used as a processing unit from input information(input data) supplied to the audio classification process unit 20 andthe video classification process unit 5 respectively. Then, at the nextstep S219, the audio classification process unit 20 and videoclassification process unit 5 carry out processing to determinesimilar-picture segments on the basis of detected characteristics andclassify the determined similar-picture segments.

Subsequently, at the next step S220, on the basis of results ofprocessing carried out by the audio classification process unit 20 andthe video classification process unit 5, the system controller 17identifies summary-information portions to be used in a digestreproduction process and reports the summary-information portions to themultiplexing unit 6. The multiplexing unit 6 adds identification to amultiplexed signal of the recognized signal segment. Then, at the nextstep S221, the principal information with identification added theretoare recorded onto the first recording medium 8 by way of the firstrecording process unit 7.

Then, at the next step S222, the system controller 17 determines whetheror not a command to stop the recording process has been received fromthe user. If the system controller 17 determines that a command to stopthe recording process has been received from the user, the execution ofthe operations represented by the flowchart shown in FIGS. 20 to 22 isended. Then, the recording/reproduction apparatus 300 typically enters astate of waiting for the user to enter a next command. If thedetermination result obtained in the operation carried out at the stepS222 indicates that a command to stop the recording process was notreceived from the user, on the other hand, the flow of the operationsgoes on to a step S223 to determine whether or not the process to recordthe desired broadcast program has been ended from, for example, theadvent of the recording end time or the arrival of another program code.

If the determination result obtained in the operation carried out at thestep S223 indicates that the process to record the desired broadcastprogram has been ended, the execution of the operations represented bythe flowchart shown in FIGS. 20 to 22 is ended. Then, therecording/reproduction apparatus 300 typically enters a state of waitingfor the user to enter a next command. If the determination resultobtained in the operation carried out at the step S223 indicates thatthe process to record the desired broadcast program has not been ended,on the other hand, the flow of the operations goes on to a step S224 todetermine whether or not the user has entered a command to change theoperating mode.

If the determination result produced in the processing carried out atthe step S224 indicates that the user has entered a command to changethe operating mode, the flow of the operations goes back to the stepS201 of the main flowchart shown in FIG. 20 to repeat the operationsdescribed above from this step. If the determination result produced inthe processing carried out at the step S224 indicates that the user hasnot entered a command to change the operating mode, on the other hand,the flow of the operations goes to the step S225 to determine whether ornot the processing to copy all the principal information determined atthe step S211 to be principal information having the oldest date fromthe first recording medium 8 to the second recording medium 42 has beencompleted.

If the determination result produced in the processing carried out atthe step S225 indicates that the processing to copy all the principalinformation having the oldest date has been completed, the flow of theoperations goes back to the step S218 to repeat the operations describedabove from this step in order to continue the process of recording newprincipal information onto the first recording medium 8. If thedetermination result produced in the processing carried out at the stepS225 indicates that the processing to copy all the principal informationhaving the oldest date has not been completed, on the other hand, theflow of the operations goes back to the step S212 to repeat theoperations described above from this step in order to continue theprocess of copying the principal information having the oldest date tothe second recording medium till completion so that new principalinformation can be recorded onto the first recording medium 8.

If the determination result produced in the operation carried out at thestep S201 indicates that the requested process is a reproductionprocess, the flow of the operations goes on to a step S226 of anothercontinuation flowchart shown in FIG. 22. At the step S226, the systemcontroller 17 receives an input of selecting a program from the user.This input selects a program stored in the first recording medium 8 (orthe main recording medium) or a program stored in the second recordingmedium 42 (or the sub recording medium) and selects an ordinaryreproduction process or a digest reproduction process.

Then, the flow of the operations goes on to a step S227 to determinewhether a program stored in the first recording medium 8 or a programstored in the second recording medium 42 is to be reproduced. If thedetermination result produced by the processing carried out at the stepS227 indicates that a program stored in the first recording medium 8 isto be reproduced, the flow of the operations goes on to a step S228. Areproduction mode to reproduce a program from the first recording medium8 is set. Then, the first reproduction control unit 9 is controlled toread out principal or summary information of the selected program fromthe first recording medium 8 and reproduce the information by adoptionof a selected method.

Subsequently, the flow of the operations goes on to a step S229 at whichthe system controller 17 determines whether or not the user has entereda command to stop the reproduction process. If the result of thedetermination indicates that the user has entered a command to stop thereproduction process, the execution of the operations represented by theflowchart shown in FIGS. 20 to 22 is ended. Then, therecording/reproduction apparatus 300 typically enters a state of waitingfor the user to enter a next command. If the determination resultproduced by the processing carried out at the step S229 indicates thatthe user has not entered a command to stop the reproduction process, onthe other hand, the flow of the operations goes on to a step S230 todetermine whether or not a command to change the operating mode has beenreceived.

If the determination result produced by the processing carried out atthe step S230 indicates that a command to change the operating mode hasbeen received, the flow of the operations goes back to the step S201 ofthe main routine shown in FIG. 20 to repeat the operations describedabove from this step. If the determination result indicates that acommand to change the operating mode has not been received, on the otherhand, the flow of the operations goes on to a step S231 to determinewhether or not a command to change the program being reproduced toanother one has been received.

If the determination result produced by the processing carried out atthe step S231 indicates that a command to change the program beingreproduced to another one has not been received, the flow of theoperations goes back to the step S228. The operations starting from thisstep as described above are repeated to continue the process ofreproducing the current program. If the determination result indicatesthat a command to change the program being reproduced to another one hasbeen received, on the other hand, the flow of the operations goes backto the step S226. The operations starting from this step as describedabove are repeated to start a process of reproducing a new program.

If the determination result produced by the processing carried out atthe step S227 indicates that a program stored in the second recordingmedium 42 is to be reproduced, on the other hand, the flow of theoperations goes on to a step S232. A reproduction mode to reproduce aprogram from the second recording medium 42 is set. Then, the secondreproduction control unit 43 is controlled to read out principalinformation of the selected program from the second recording medium 42and carry out an ordinary reproduction process.

Subsequently, at the next step S233, the system controller 17 determineswhether or not the user has entered a command to stop the reproductionprocess. If the result of the determination indicates that the user hasentered a command to stop the reproduction process, the execution of theoperations represented by the flowchart shown in FIGS. 20 to 22 isended. Then, the recording/reproduction apparatus 300 typically enters astate of waiting for the user to enter a next command. If thedetermination result produced by the processing carried out at the stepS233 indicates that the user has not entered a command to stop thereproduction process, on the other hand, the flow of the operations goeson to a step S234 to determine whether or not a command to change theoperating mode has been received.

If the determination result produced by the processing carried out atthe step S234 indicates that a command to change the operating mode hasbeen received, the flow of the operations goes back to the step S201 ofthe main routine shown in FIG. 20 to repeat the operations describedabove from this step. If the result of the determination indicates thata command to change the operating mode has not been received, on theother hand, the flow of the operations goes on to a step S235 todetermine whether or not a command to change the program beingreproduced to another one has been received.

If the determination result produced by the processing carried out atthe step S235 indicates that a command to change the program beingreproduced to another one has not been received, the flow of theoperations goes back to the step S232. The operations starting from thisstep as described above are repeated to continue the process ofreproducing the principal information of the current program. If theresult of the determination indicates that a command to change theprogram being reproduced to another one has been received, on the otherhand, the flow of the operations goes back to the step S226. Theoperations starting from this step as described above are repeated tostart a process of reproducing a new program.

<Summary of the Third Embodiment>

As described above, when the residual storage capacity of the firstrecording medium 8 becomes smaller than a value determined in advance,principal information of the oldest date is moved from the firstrecording medium 8 to the second recording medium 42 to increase theresidual storage capacity of the first recording medium 8 before newprincipal information is stored onto the first recording medium 8. Thus,a problem of an inability to store principal information of a newprogram due to the fact that the residual storage capacity of the firstrecording medium 8 becomes insufficient can be prevented from arising.

In addition, the summary information of a program that has the principalinformation moved to the second recording medium 42 is deliberately leftin the first recording medium 8. Therefore, a digest reproductionprocess of the summary information of the program having the oldest datecan be carried out at any time quickly. On top of that, an ordinaryreproduction process of the principal information of the new program anda digest reproduction process of the summary information of the programcan also be carried out at any time quickly. That is to say, the meritsof the accessibility characteristic of the first recording medium 8 donot deteriorate.

Moreover, management to rearrange and preserve information signalsstored in the recording medium can be made simpler, making it possibleto prepare an environment of promoting effective and efficient use of alarge number of presented various video and audio signals.

It is to be noted that, in the recording/reproduction apparatus 300implemented by the third embodiment, additional identifications are alsorecorded so as to allow summary information created for principalinformation to be stored in a recording medium by being associated withthe principal information. Thus, the third embodiment allows the summaryinformation to be extracted even after a recording process. However, thepresent invention is by no means limited to such a scheme.

For example, principal information and summary information are bothrecorded on the first recording medium 8 as is the case with therecording/reproduction apparatus 200 implemented by the secondembodiment described earlier. When the residual storage capacity of thefirst recording medium 8 becomes insufficient, only principalinformation of the oldest date is moved from the first recording medium8 to the second recording medium 42.

In the third embodiment described above, only principal information ofone program having the oldest date is moved from the first recordingmedium 8 to the second recording medium 42. However, principalinformation of a plurality of programs in ascending order of dates canalso be moved from the first recording medium 8 to the second recordingmedium 42. For example, principal information of two or three programsin ascending order of dates can also be moved from the first recordingmedium 8 to the second recording medium 42.

In addition, more than one piece of principal information can be movedsequentially one piece after another starting with a piece having theoldest date till a predetermined residual storage capacity of the firstrecording medium 8 is preserved.

On top of that, as described earlier, the processing to transferprincipal information from the first recording medium 8 to the secondrecording medium 42 can be move processing or copy processing. In thecase of the move processing, after principal information is transferredfrom the first recording medium 8 to the second recording medium 42, theprincipal information left in the first recording medium 8 as anunnecessary portion is erased from the first recording medium 8 to newlyallocate a free area.

In the case of the copy processing, on the other hand, after principalinformation is transferred from the first recording medium 8 to thesecond recording medium 42, the principal information left in the firstrecording medium 8 as an unnecessary portion is put in status ofallowing new principal information to be overwritten thereon.

In addition, as described earlier, summary information is deliberatelylift in the first recording medium 8. If the summary informationassociated with the transferred principal information is left atscattered locations in the first recording medium 8, the recording areaof the first recording medium 8 is put in a state of being worm-eaten.In order to prevent the recording area from being put in a state ofbeing difficult to use, a so-called defragmentation process is may becarried out to make the recording area of the first recording medium 8usable effectively.

On top of that, a message can be displayed to the user through thedisplay system 33 to inform the user that the residual storage capacityof the first recording medium 8 has become too small, the residualstorage capacity of the second recording medium 42 has become too small,principal information has been or is being transferred from the firstrecording medium 8 to the second recording medium 42 or adefragmentation process has been or is being carried out to prevent therecording area of the first recording medium 8 from being put in a stateof being worm-eaten, or report other facts to the user.

In addition, the second recording medium 42 having a storage capacitygreater than that of the recording medium 8 is desirable.

Fourth Embodiment

FIG. 23 is an explanatory block diagram showing a recording/reproductionapparatus 400 implemented by a fourth embodiment to which an embodimentof the information-signal process apparatus and information-signalprocess method provided by the present invention is applied. FIGS. 24and 25A to 25C are explanatory diagrams showing an principle ofoperations carried out by the recording/reproduction apparatus 400implemented by the fourth embodiment.

Much like the recording/reproduction apparatus 100 implemented by thefirst embodiment, the recording/reproduction apparatus 400 implementedby the fourth embodiment is capable of creating shrunk pictures (orthumb-nail pictures) from a video signal of a television broadcastprogram with predetermined timings in a process of recording the videoand audio signals of the program onto a recording medium and displayingthe created thumb-nail pictures in a tabular format. Therecording/reproduction apparatus 400 implemented by the fourthembodiment is different from the recording/reproduction apparatus 100implemented by the first embodiment, however, in that therecording/reproduction apparatus 400 adopts a different display formatof shrunk pictures from that of the recording/reproduction apparatus100.

Even though the recording/reproduction apparatus 400 implemented by thefourth embodiment has all but the same configuration as that of therecording/reproduction apparatus 100 implemented by the firstembodiment, in order to clarify differences in operation between theseembodiments, a block diagram of the recording/reproduction apparatus 400implemented by the fourth embodiment is shown in FIG. 23. By referringto the block diagram of FIG. 23, the recording/reproduction apparatus400 implemented by the fourth embodiment is explained in detail asfollows.

<Principle of Operation>

First of all, the principle of operation of the recording/reproductionapparatus 400 implemented by the fourth embodiment is described. Therecording/reproduction apparatus 400 shown in FIG. 23 receives analogvideo and audio signals of typically a television broadcast program asan information signals. The recording/reproduction apparatus 400 thenconverts the analog video and audio signals into digital data, andcompresses the digital data by adoption of an MPEG method. Subsequently,the compressed digital video data is time-division multiplexed with thecompressed digital audio data to be recorded onto the recording medium8. The recording medium 8 is a disk-shaped recording medium having alarge storage capacity. An example of such a disk-shaped recordingmedium is a hard disk.

In a reproduction process, the recording/reproduction apparatus 400shown in FIG. 23 reads out the multiplexed signal from the recordingmedium 8, separating video and audio data of the multiplexed signal fromeach other. The video and audio data is then decompressed to recover thepre-compression video and audio data. Subsequently, the video and audiodata obtained as a result of the decompression is converted into analogoutput signals.

In addition, the recording/reproduction apparatus 400 shown in FIG. 23classifies pictures of the video signal into similar pictures in aprocess to record the video and audio signals onto the recording medium8. Much like the recording/reproduction apparatus 100 implemented by thefirst embodiment, the classification processing is carried out on MPEGvideo data, which is compressed video data. Similar pictures aredetected in I-picture units and I pictures similar to each other arecategorized into a similar-picture segment.

FIG. 24A is a diagram showing a series of input video signals composinga news program taken as an example. The series of input video signalscomprises 31 I pictures, which are classified into similar-picturesegments A1, A2 and A3 each consisting of similar I pictures.

In this case, I pictures included in each of the similar-picturesegments A1, A2 and A3 are pictures, which show a news caster readingnews and, hence, display relatively few changes in news-caster motion.With only such similar-picture scenes, nevertheless, the contents of theprogram can be conceivably grasped with a relatively high degree ofefficiency. Further consideration of the similar-picture segments A1, A2and A3 also indicates that, in many cases, the head of each of thesimilar-picture segments A1, A2 and A3 is conceivably a picture showinga scene in which the news caster starts reading a news article. The headpicture of each of the similar-picture segments A1, A2 and A3 generallydisplays a news telop (or a news headline) at the bottom of the screen.

For the reason described above, shrunk forms of I pictures eachpositioned at the head of a similar-picture segment of a news program orat a location in close proximity to the head are created as shown inFIG. 24B. Then, with the lapse of time, created shrunk pictures aredisplayed sequentially in a chronological order as pictures 1 a to 1 gshown in FIG. 25A. In such a screen, a news telop is also conceivablydisplayed in each of the pictures 1 a to 1 g in many cases so that, bymerely viewing this tabular display (a multi-picture display) of shrunkpictures, the user can quickly and reliably know what the contents ofthe news are.

If a news program is broadcasted again after the lapse of apredetermined time period, the same processing carried out on theimmediately preceding news program is carried out again on the currentnews program to generate shrunk pictures 1 a to 1 h shown in FIG. 25B.The shrunk pictures 1 a to 1 g shown in FIG. 25A as shrunk pictures ofthe immediately preceding news program are displayed again on the samescreen as pictures 2 a to 2 g shown in FIG. 25B.

Assume that 8 news programs are broadcasted sequentially and the sameprocessing as that described above is carried out again on thesubsequent news programs. In this case, the 8 news programs aredisplayed on horizontal rows 1 to 8 of the screen respectively as shownin FIG. 25C. On each of columns a to h of every horizontal row, a shrunkpicture of a news program appearing on the row is displayed.

As described above, previous news programs each represented by shrunkpictures are displayed sequentially on lower rows arranged in thevertical direction. In this way, it is possible to easily recognizechanges of the same news with the lapse of time and new news developedas time goes by.

In addition, by selecting a desired one of shrunk pictures displayed ina tabular format as explained earlier by referring to the flowchartshown in FIGS. 8 and 9, it is possible to reproduce a signal segmentincluding an original picture corresponding to the selected shrunkpicture as an original picture positioned at the head thereof or carryout a digest reproduction process so as to let the user know detailedcontents of desired news. That is to say, both an ordinary reproductionprocess and a digest reproduction process can be carried out with a highdegree of efficiency.

It is to be noted that, in order to detect a signal of a news telop witha higher degree of accuracy, a telop may be detected along withdetection of a similar picture. Since characters composing a telop of anews program are normally displayed in a white color in many cases, thetelop is usually displayed at a telop-display position, which is aposition on a lower-side portion of the screen in most cases. Thus, byrecognizing characteristics of a luminance signal among picture signalsdisplayed at a position corresponding to the lower-side portion of thescreen, a screen, which is an I picture or a frame, displaying a telopcan be detected.

<Operations and Configuration of the Recording/Reproduction Apparatus400 Implemented by the Fourth Embodiment>

The following description explains operations and configuration of therecording/reproduction apparatus 400 implemented by the fourthembodiment shown in FIG. 23. The configuration shown in FIG. 23 can bedivided into portions on the left and right sides of the recordingmedium 8. The portion on the left side is mainly a recording systemwhile the portion on the right side is mainly a reproduction system.

<Recording System>

First of all, the recording system is explained. An analog audio signal(an audio input) supplied to the recording/reproduction apparatus 400implemented by the fourth embodiment is fed to an audio A/D converter 1for converting the analog audio signal into a digital audio signal to besupplied to an audio encoder 2 and an audio classification process unit20.

The audio encoder 2 carries out a band compression process on thedigital audio signal by adoption of a predetermined compressiontechnique such as an MPEG audio method. Audio data obtained as a resultof the band compression process carried out on the digital audio signalis supplied to a multiplexing process unit 6.

The audio classification process unit 20 detects continuity of afrequency spectrum peak in the digital audio signal typically for eachpredetermined segment to determine whether or not the digital audiosignal is a musical signal. The result of the determination is suppliedto the multiplexing process unit 6, a system controller 17 and apicture-scene classification display and reproduction-position displayunit 19. The picture-scene classification display andreproduction-position display unit 19 is also properly referred tohereafter as a classification display unit.

It is to be noted that, instead of processing the digital audio signalreceived from the audio A/D converter 1, the audio classificationprocess unit 20 can also be designed into a configuration for receivingthe audio data, which is obtained as a result of the band compressionprocess carried out by the audio encoder 2 on the digital analog signalgenerated by the audio A/D converter 1, as indicated by a dashed lineshown in FIG. 23, and carrying out the classification processingdescribed earlier on the audio data.

On the other hand, an analog video signal (a video input) supplied tothe recording/reproduction apparatus 400 implemented by the fourthembodiment is fed to a video A/D converter 3 for converting the analogvideo signal into a digital video signal to be supplied to a videoencoder 4.

The video encoder 4 carries out a band compression process on thedigital video signal by adoption of a predetermined compressiontechnique such as an MPEG2 method. Video data obtained as a result ofthe band compression process carried out on the digital video signal issupplied to a video classification process unit 5, the multiplexingprocess unit 6, an I-picture detection unit 21 and a shrunk-pictureprocess unit 22.

The video classification process unit 5 detects picture characteristicsof I pictures of the video data received from the video encoder 4 andrecognizes picture similarity on the basis of the detectedcharacteristics. In addition, the video classification process unit 5also recognizes similar-picture segments as explained earlier byreferring to FIG. 24. The video classification process unit 5 alsogenerates an identification information signal for identifying thesimilar-picture segments and supplies the signal to the multiplexingprocess unit 6 and the classification display unit 19. The signalidentification information includes position information of the videodata and the position information of the video data includes informationon how many consecutive similar frames are included in eachsimilar-picture segment.

It is to be noted that, instead of receiving the compressed video datafrom the video encoder 4, the video classification process unit 5 mayalso receive the digital video signal output by the video A/D converter3 as indicated by a dashed line shown in FIG. 23 and carries out theprocesses to identify and classify similar pictures of the digital videodata in frame units as described above on the basis of the digital videosignal.

In addition, it is also possible to provide a configuration in which theidentification information signals obtained as results of the processesto classify the video and audio signals can also be generated by thesystem controller 17 on the basis of the classification-process resultsreceived from the video classification process unit 5 and the audioclassification process unit 20 and supplied to the multiplexing processunit 6.

As is the case with the recording/reproduction apparatus 100 implementedby the first embodiment described earlier, the system controller 17 is amicrocomputer comprising a CPU (Central Processing Unit), a ROM (ReadOnly Memory), a RAM (Random Access Memory), a non-volatile memory and atimer circuit, which are used for controlling the each componentscomposing the recording/reproduction apparatus 400.

In addition, an I-picture detection unit 21 in the configuration shownin FIG. 23 is a component for detecting I pictures from the video dataand reporting detection information such as detection timings to thesystem controller 17. Only I pictures are detected because, as describedabove, the recording/reproduction apparatus 400 implemented by thisembodiment carries out the picture-similarity detection process and thepicture classification process in I-picture units. The system controller17 is capable of controlling the video classification process unit 5 andthe shrunk-picture process unit 22 on the basis of the detectioninformation received from the I-picture detection unit 21.

In accordance with the control executed by the system controller 17, theshrunk-picture process unit 22 generates shrunk-picture data from videodata (to be more specific, I pictures each located at the head of asimilar-picture segment or at a location in close proximity to the head)received thereby and supplies the data to the multiplexing process unit6 and the classification display unit 19.

In this fourth embodiment, the classification display unit 19 receivesthe classification result from the audio classification process unit 20,the identification information signal from the video classificationprocess unit 5 and the shrunk-picture data from the shrunk-pictureprocess unit 22, displaying shrunk pictures each located at the head ofa similar-picture segment in a chronological order in the format shownin FIGS. 25A to 25C. That is to say, the recording/reproductionapparatus 400 implemented by this embodiment is capable of displayingthumb-nail pictures for recorded original pictures concurrently with theprocess to record the original pictures onto the recording medium.

In addition, the multiplexing process unit 6 generates multiplexed databy typically carrying out a time-division multiplexing process on piecesof information including the audio data, the video data, theshrunk-picture data and the identification information signal receivedfrom the video classification process unit 5, supplying the multiplexeddata to the recording process unit 7. The recording process unit 7carries out processing including a process to add error correction codesto the multiplexed data and a process to interleave the data to create asignal to be recorded, recording the signal into a predetermined area inthe recording medium 8.

Then, in the case of this embodiment, as described before, by using theidentification information signal multiplexed in the multiplexed data,it is possible to identify similar-picture segments and associateoriginal pictures with shrunk pictures representing the originalpictures. Thus, it is possible to display shrunk pictures, each locatedat the head of a similar-picture segment in a chronological order in theformat shown in FIGS. 25A to 25C during a recording process or at arecording-end time or with a predetermined timing after the end of therecording process.

It is to be noted that, much like the first embodiment explained earlierby referring to FIGS. 4 to 6, the video classification process unit 5detects characteristics of video data for each I picture, which is usedas a processing unit, recognizes similar pictures on the basis of thedetected characteristics and determines similar-picture segments eachcomprising pictures similar to each other. Also as described earlier,the same identification is assigned to similar pictures so that, byusing such an identification, it is possible to identify which range ofpictures is included in each similar-picture segment.

In addition, much like the first embodiment explained earlier byreferring to FIG. 7, the audio classification process unit 20 recognizessimilarity of sounds for each predetermined time unit and, much like thesecond and third embodiments, the audio classification process unit 20also detects audio levels.

<Reproduction System>

Next, the reproduction system is explained. A control signal generatedby the system controller 17 controls the reproduction control unit 9 toread out multiplexed data specified by the user from the recordingmedium 8 and supply the data to the reproduction process unit 10. Thereproduction process unit 10 carries out predetermined processesincluding deinterleave processing on the multiplexed data receivedthereby and supplies the processed multiplexed data to the separationprocess unit 11. The separation process unit 11 demultiplexes themultiplexed data supplied thereto to generate audio data, video data,shrunk-picture data and an identification information signal to mentiona few.

The audio data generated by the separation process unit 11 is suppliedto an audio decoder 12. The audio data supplied to the audio decoder 12is data compressed by adoption of a predetermined compression method.The audio decoder 12 thus decompresses the compressed audio datareceived thereby and supplies audio data obtained as a result of thedecompression to an audio digital/analog converter 13, which isabbreviated hereafter to an audio D/A converter.

The audio D/A converter 13 converts the digital audio signal suppliedthereto into an analog audio signal and outputs the analog audio signalto a speaker. The speaker generates a sound according to the analogaudio signal supplied thereto.

In addition, the video data generated by the separation process unit 11is supplied to a video decoder 14. The video data supplied to the videodecoder 14 is data compressed by adoption of a predetermined compressionmethod. The video decoder 14 thus decompresses the compressed video datareceived thereby and supplies video data obtained as a result of thedecompression to a video digital/analog converter 15, which isabbreviated hereafter to a video D/A converter.

The video D/A converter 15 converts the digital video signal suppliedthereto into an analog video signal and outputs the analog video signalto a monitor receiver. The monitor receiver displays a picture accordingto the analog video signal supplied thereto.

The identification information signal separated by the separationprocess unit 11 is supplied to an identification-signal detection unit16. The identification-signal detection unit 16 detects necessaryidentification information signal from the identification informationsignal received thereby and supplies the identification informationsignal to the system controller 17 and the classification display unit19.

In addition, the identification-signal detection unit 16 also extracts apredetermined timestamp signal or frame numbers from the identificationinformation signal received thereby and supplies the timestamp signal orthe frame numbers to the system controller 17. By detecting areproduction point of time on the basis of the timestamp signal, areproduction position can also be displayed on the classificationdisplay unit 19.

Then, when a command is entered via the user input I/F 18 as a commandmaking a request for a display classifying a broadcast program recordedon the recording medium 8, the system controller 17 controls thereproduction control unit 9 on the basis of the identificationinformation signal supplied to the system controller 17 to read outshrunk pictures of the desired broadcast program from the recordingmedium 8 and supplies the shrunk pictures to the classification displayunit 19 by way of the reproduction process unit 10 and the separationprocess unit 11. In addition, the system controller 17 controls theclassification display unit 19 on the basis of the identificationinformation signal to display a classification of the broadcast programrecorded on the recording medium 8 as shown in FIGS. 25A to 25C.

As described above, even with a predetermined timing after the end ofthe recording process, by using the identification information signal aswell as the shrunk-picture signal, which have been generated on thebasis of the picture-similarity determination processing and then storedin the recording medium 8, a classification of pictures can be displayedas shown in FIGS. 25A to 25C.

It is to be noted that, this embodiment has been exemplified by giving acase of recording a news program as an example. However, the embodimentis by no means limited to such a case. For example, the embodiment canalso be applied to a case of recording a musical program or the likeonto the recording medium. In the case of recording a musical programonto the recording medium, scenes each showing a singer singing a songare classified. A scene showing a desired singer singing a song can thusbe found fast from the recording medium and, for example, a programportion including such a scene can therefore be reproduced quickly inthe normal mode.

In addition, the processing described above can be carried out in thesame way also for a case of recording a sport program such as a baseballof a tennis program. That is to say, a picture located at the head ofeach similar-picture segment is displayed. In the case of a baseballprogram, the picture located at the head of a similar-picture segmentmay be a scene showing a butter entering a butter box or a scene showinga pitcher throwing the ball. In the case of a tennis tournament, apicture located at the head of each similar-picture segment can be usedfor searching the tennis program for a serve-scene.

On top of that, it is also possible to generate picture data for adigest reproduction process and picture data for a skimming reproductionprocess during a recording process and store these pieces of data alongwith the main picture data of the broadcast program so that the digestand skimming reproduction processes of the broadcast program can becarried out.

If it is desired to carry out a special reproduction process such as thedigest or skimming reproduction process, the user needs to enter acommand making a request for the special reproduction process to therecording/reproduction apparatus 400 via the user input I/F 18.Receiving the command entered by the user, the system controller 17controls the reproduction control unit 9 to read out picture data forthe desired special reproduction method so as to allow pictures of thedesired special reproduction method to be reproduced.

<Operations of the Recording/Reproduction Apparatus 400>

Next, operations carried out by the recording/reproduction apparatus 400implemented by the fourth embodiment described above in a recordingprocess are explained by referring to a flowchart shown in FIG. 26. FIG.26 shows a flowchart referred to in explanation of operations carriedout by the recording/reproduction apparatus 400 implemented by thefourth embodiment.

The processing represented by the flowchart shown in FIG. 26 is carriedout when the system controller 17 receives a command entered by the userto carry out processes to record for example a news program and todisplay classifications of the program.

The flowchart begins with a step S301, at which the system controller 17sets start and end times of one or more news programs as recording starttimes and recording end times respectively in a non-volatile memory ofthe system controller 17 on the basis of for example an EPG (ElectronicProgram Guide) presented as a guide included in a broadcast signal of adigital television broadcast.

In the operation carried out at the step S301, start and end times ofone or more news programs can also be received as inputs from the userinput I/F and set as recording start times and recording end timesrespectively. That is to say, recording start times and recording endtimes can also be set manually by the user.

It is to be noted that the system controller 17 includes a clock circuitnot shown in FIG. 23. The clock circuit provides the present data, thepresent day of the week and the present time.

Then, at the next step S302, the system controller 17 determines whetheror not the user has entered a command to stop the process to record anews program for example onto the recording medium and stop the processto exhibit a classification display.

If the determination result produced by the operation carried out at thestep S302 indicates that the user has not entered a stop command, theflow of the operations goes on to a step S303 to determine whether ornot the present time computed by the clock circuit has attained therecording start time of the current news program.

If the determination result produced by the operation carried out at thestep S303 indicates that the present time computed by the clock circuithas not attained the recording start time of the current news program,the flow of the operations goes on to a step S304 to enter a wait state(or a standby state) by repeating the operations described above fromthe step S302.

If the determination result produced by the operation carried out at thestep S303 indicates that the present time computed by the clock circuithas attained the recording start time of the current news program, onthe other hand, the flow of the operations goes on to a step S305 atwhich the system controller 17 controls components such as the videoclassification process unit 5 and the I-picture detection unit 21 todetect characteristics of video data of a news program to be recordedonto the recording medium. Then, at the next step S306, an operation toidentify similar pictures is carried out.

It is to be noted that, in the operations carried out at the steps S305and S306, the audio classification process unit 20 may also becontrolled to recognize similar segments in the audio signal to berecorded onto the recording medium, and such segments can also be takeninto consideration. For example, segments including similar pictures andsimilar sounds can each be set as a similar segment.

Then, at the next step S307, the system controller 17 also controls theshrunk-picture process unit 22 to carry out a thumb-nail operation ofcreating shrunk pictures from I pictures each located at the head of asimilar-picture segment. Subsequently, at the next step S308, the systemcontroller 17 determines whether or not the present time computed by theclock circuit has attained the recording end time of the current newsprogram.

If the determination result produced by the operation carried out at thestep S308 indicates that the present time computed by the clock circuithas attained the recording end time of the current news program, theflow of the operations goes on to a step S309 at which the systemcontroller 17 controls the classification display unit 19 to show atabular display of shrunk pictures like those shown in FIGS. 25A to 25Cin the multi-picture display format.

If the determination result produced by the operation carried out at thestep S308 indicates that the present time computed by the clock circuithas not attained the recording end time of the current news program, onthe other hand, the flow of the operations goes on to a step S310. Theflow of the operations also goes on to a step S310 after the operationcarried out at the step S309 has been completed. At the step S310, thesystem controller 17 determines whether or not the user has entered acommand to stop the process to record a news program onto the recordingmedium and stop the process to exhibit a classification display.

If the determination result produced by the operation carried out at thestep S310 indicates that the user has not entered a stop command, theflow of the operations goes back to the step S303 at which theoperations described above are repeated from this step to record a newsprogram received next and carry out the classification display operationfor this news program. If the determination result produced by theoperation carried out at the step S310 indicates that the user hasentered a stop command, on the other hand, the execution of theoperations represented by the flowchart shown in FIG. 26 is ended. Then,the recording/reproduction apparatus 400 typically enters a state ofwaiting for the user to enter a next command.

<Summary of the Fourth Embodiment>

As described above, the recording/reproduction apparatus 400 implementedby the fourth embodiment is capable of identifying similar-picturesegments of a video signal and an audio signal of a television broadcastprogram in a recording process, displaying pictures each located at thehead of one of the similar-picture segments as shrunk pictures (orthumb-nail pictures) in a tabular display format (or a multi-picturedisplay format) and updating the tabular display screen by adding anarray of shrunk pictures representing a new program on the top row ofthe screen. Thus, contents of a plurality of programs recorded so farcan be grasped effectively and with a high degree of efficiency to showcontents always updated with the lapse of time.

In addition, since only some data of an information signal can bedisplayed in a tabular format instead of displaying the entire data ofthe signal, contents of the information signal can be grasped with ahigh degree of accuracy and a high degree of precision in a short periodof time.

On top of that, by specifying a shrunk picture, a sequence of picturesstarting with a picture corresponding to the shrunk picture can benormally reproduced as a picture sequence beginning at a start point oftime corresponding to the shrunk picture.

As described above, since it is possible to identify similar-picturesegments, display pictures in a tabular display format and update thetabular display screen every time a new program is processed, processingsuch as an operation to grasp contents of programs and an operation tospecify a reproduction point of time can be carried out quickly andaccurately.

In addition, since shrunk pictures are also stored on a recordingmedium, the stored shrunk pictures can be displayed in a tabular formatshown in FIGS. 25A to 25C at any time. Then, a portion represented byone of the shrunk pictures can be reproduced as a desired portion of theinformation signal represented.

On top of that, if it is impossible to accommodate a classificationdisplay in a screen, the classification display can be scrolled in thehorizontal direction or the vertical direction so as to make the usercapable of viewing all shrunk pictures.

Furthermore, in the case of the fourth embodiment, a video signal is thefirst information signal while shrunk pictures (thumb-nail pictures)created as a result of shrinking predetermined I pictures of the videosignal correspond to the second information signal. As described above,the second information signal can be any signal as long as the signal isobtained as a result of carrying out a predetermined process onpredetermined portions of the first information signal. Thus, the secondinformation signal can be generated as thumb-nail pictures each obtainedas a result of shrinking picture data contained in 1 frame of apredetermined portion of the video signal.

Moreover, the first information signal is by no means limited to a videosignal. For example, the first information signal can be an audiosignal. If an audio signal is used as the first information signal, forexample, a message created in the fourth embodiment can be used as thesecond information signal. As another example, text data or the like canbe used as the first information signal.

Fifth Embodiment

FIG. 27 is an explanatory block diagram showing a recording/reproductionapparatus 500 implemented by fifth embodiment to which an embodiment ofthe information-signal process apparatus and information-signal processmethod provided by the present invention is applied. FIGS. 28A to 28Cand 29A to 29C are explanatory diagrams showing an outline of operationscarried out by the recording/reproduction apparatus 500 implemented bythe fifth embodiment.

The recording/reproduction apparatus 500 implemented by the fifthembodiment makes video information of similar-picture segmentsextractable from principal information of a television broadcast programand makes the extracted video information of similar-picture segmentsusable in advance during a process to record the television broadcastprogram onto a recording medium, or creates summary information to beadded to the principal information in advance during the recordingprocess. Thus, the so-called digest reproduction process can be carriedout.

In addition, the recording/reproduction apparatus 500 implemented by thefifth embodiment is provided with a plurality of digest reproductionmethods and capable of selecting a proper one of the methods as a methodto be adopted in a digest reproduction process in accordance with thegenre of a recorded television broadcast program or capable of selectinga proper one of the methods as a method to prevent the reproduction timefrom becoming long.

<Principle of Operation>

The following description explains the principle of operation of therecording/reproduction apparatus 500 implemented by the fifthembodiment. Much like the recording/reproduction apparatus implementedby the embodiments described earlier, the recording/reproductionapparatus 500 shown in FIG. 27 is capable of converting an analoginformation signal, which is typically analog video and audio signals ofa television broadcast program, into digital data, compressing thedigital data, multiplexing the compressed digital data and recording themultiplexed compressed digital data onto the recording medium 8. Therecording medium 8 is typically a disk-shaped recording medium having alarge storage capacity. An example of such a disk-shaped recordingmedium is a hard disk.

In addition, the recording/reproduction apparatus 500 shown in FIG. 27is capable of reading out a multiplexed compressed digital signal fromthe recording medium 8, demultiplexing the multiplexed compresseddigital signal read out from the recording medium 8 to generatecompressed digital video and audio data separated from each other,decompressing the compressed digital video and audio data to recover thepre-compression digital video and audio data, converting thepre-compression digital video and audio data into analog video and audiosignals and outputting the analog video and audio signals.

Much like the recording/reproduction apparatus implemented by theembodiments described earlier, the recording/reproduction apparatus 500shown in FIG. 27 is also capable of recognizing I pictures similar toeach other, classifying similar pictures into similar-picture segments,adding an identification information to similar pictures so as to makeeach of the similar-picture segments recognizable and recording thesimilar-picture segments onto the recording medium.

In addition, the recording/reproduction apparatus 500 implemented by thefifth embodiment basically discriminates similar pictures included in aninput picture series, identifies similar-picture segments(similar-picture scenes) and makes the identified similar-picturesegments usable in a digest reproduction process.

As will be described below, however, the recording/reproductionapparatus 500 implemented by the fifth embodiment has four digestreproduction modes A, B, C and D. These digest reproduction modes areexplained below by taking a process to record an input picture sequenceshown in FIG. 28A onto a recording medium as an example.

In the input picture sequence shown in FIG. 28A, a similar-picturesegment A1 includes pictures 3, 4 and 5 similar to each other. By thesame token, a similar-picture segment A2 includes pictures 13, 14, 15and 16 similar to each other. In the same way, a similar-picture segmentA3 includes pictures 22, 23, 24, 25 and 26 similar to each other.

In such a case, the digest reproduction mode A is a mode forsequentially reproducing the similar-picture segments A1, A2 and A3along the time series as shown in FIG. 28B. The digest reproduction modeA shown in FIG. 2B is effective for a case in which contents of aprogram, which has relatively few changes, can be grasped with arelatively high degree of efficiency by viewing only similar-picturescenes in the program. Examples of the scenes are a picture scene of anews caster in the case of a news program and a picture scene ofspecific performers in the case of a talk-show program.

A picture segment B1 includes the similar-picture segment A1 andsegments immediately preceding and succeeding the similar-picturesegment A1. By the same token, a picture segment B2 includes thesimilar-picture segment A2 and segments immediately preceding andsucceeding the similar-picture segment A2. In the same way, a picturesegment B3 includes the similar-picture segment A3 and segmentsimmediately preceding and succeeding the similar-picture segment A3. Thedigest reproduction mode B is a mode for sequentially reproducing thepicture segments B1, B2 and B3 along the time series as shown in FIG.28C. The digest reproduction mode B is effective for a case in which thesimilar-picture segment is for example a batting scene in the case of abaseball game program or a serving scene in the case of a tennistournament program.

That is to say, if continuity exists throughout a program as is the casewith certain kinds of sport program including a baseball game programand a tennis tournament program, by displaying also segments immediatelypreceding and succeeding each similar-picture segment along with thesimilar-picture segments rather than merely displaying only thesimilar-picture segments, the segments immediately preceding andsucceeding each similar-picture segment can be used for indicating thecontext of each similar-picture segment so that contents of the programcan be grasped effectively and with a high degree of efficiency withouta sense of incompatibility.

In addition, it is possible to acquire information allowing the genre ofa television broadcast program to be recorded, that is, informationindicating whether the program is a news program, a sport program or aprogram of another type, from a presented EPG data included in thesignal of the television broadcast program and also to store theacquired information onto the recording medium in advance so that adigest reproduction mode optimum for the genre of a program to bereproduced can be selected automatically in a digest reproductionprocess. Thus, the digest reproduction process can be carried out in theoptimum digest reproduction mode.

It is needless to say that the user itself is allowed to specify apreferred digest reproduction mode to be used in a digest reproductionprocess so that the digest reproduction process can be carried out inthe selected digest reproduction mode. In addition, the processing todetect similar pictures and the processing to determine similar-picturesegments can be carried out and implemented in the same way as theprocessing performed by the picture-classification process unit employedin each of the other embodiments described earlier.

The descriptions of the embodiments are focused on similarity ofpictures. However, attention can also be paid to attributes of an audiosignal. Typically, an audio signal is classified into musical-soundsegments and conversational-voice segments and the digest reproductionmode A or B is adopted.

In this case, for example, in the digest reproduction mode A of amusical program, only scenes are played back in a digest reproductionprocess to generate music, or the musical program is edited to produceonly scenes generating music in a digest reproduction process. Thus, aneffective digest reproduction process can be carried out.

In the digest reproduction mode B applied to a musical program such as apopular-song program, on the other hand, it is assumed that, forexample, introduction of a singer or a performer prior to the start ofeach song can be displayed. Thus, a digest reproduction process can becarried out on only introductory portions each preceding a song to besung as a portion introducing the song singer appearing in thepopular-song program or the program can be edited to generate only suchintroductory portions to be displayed in a digest reproduction process.Thus, an effective digest reproduction process can be carried out.

It is to be noted that the processing to detect and classifymusical-sound segments and conversational-voice segments can be carriedout and realized in the same way as the processing performed by theaudio classification process unit 20 employed in therecording/reproduction apparatus implemented by each of the otherembodiments described earlier.

By the way, in the case of the digest reproduction modes A and B shownin FIGS. 28B and 28C respectively, if similar-picture segments are toolong, the reproduction time is also prolonged excessively, conceivablygiving rise also to a case, in which contents of a recorded programcannot be grasped in a short period of time with a high degree ofefficiency, in consequence.

In order to solve this problem, assume that, in an input picturesequence shown in FIG. 29A, a similar-picture segment C1 includespictures 3, 4 and 5 similar to each other, a similar-picture segment C2includes pictures 13, 14, 15 and 16 similar to each other and asimilar-picture segment C3 includes pictures 22, 23, 24, 25 and 26similar to each other as is the case with the input picture sequenceshown in FIGS. 28A to 28C. The similar-picture segments C1, C2 and C3shown in FIG. 29A are used as digest reproduction segments.

The digest reproduction mode C is a mode in which the length of eachdigest reproduction segment, that is, the length of each picture segmentused in a digest reproduction process is determined in advance as shownin FIG. 29B so that it is possible to carry out a digest reproductionprocess with a digest reproduction time not exceeding the sum of thedetermined lengths of the digest reproduction segments.

The digest reproduction mode D shown in FIG. 29C is a mode in which thedigest reproduction process of each digest reproduction segment in thedigest reproduction mode C shown in FIG. 29B is expedited to start at apoint of time leading ahead of the head of the digest reproductionsegment by a predetermined time interval. In the case of the digestreproduction mode D, a scene flow between a similar-picture segment anda picture segment preceding the similar-picture segment can also begrasped as is the case with the digest reproduction mode B. For certainprogram genres such as the genre of a sport program, a more effectivedigest reproduction process can be carried out and the digestreproduction time will never be prolonged more than necessary.

<Operations and Configuration of the Recording/Reproduction Apparatus500 Implemented by the Fifth Embodiment>

Operations and configuration of the recording/reproduction apparatus 500implemented by the fifth embodiment shown in FIG. 27 are explained. Theconfiguration shown in FIG. 27 can be divided into portions on the leftand right sides of the recording medium 8. The portion on the left sideis mainly a recording system while the portion on the right side ismainly a reproduction system.

<Recording System>

First of all, the recording system is explained. An analog audio signal(an audio input) supplied to the recording/reproduction apparatus 400implemented by the fifth embodiment is fed to an audio A/D converter 1for converting the analog audio signal into a digital audio signal to besupplied to an audio encoder 2 and an audio classification process unit20.

The audio encoder 2 carries out a band compression process on thedigital audio signal by adoption of a predetermined compressiontechnique such as an MPEG audio method. Audio data obtained as a resultof the band compression process carried out on the digital audio signalis supplied to a multiplexing process unit 6.

The audio classification process unit 20 detects continuity of afrequency spectrum peak in the digital audio signal typically for eachpredetermined segment to determine whether or not the digital audiosignal is a musical signal. The result of the determination is suppliedto the multiplexing process unit 6, a system controller 17 and apicture-scene classification display and reproduction-position displayunit 19, which is a unit for showing a picture scene classificationdisplay and reproduction position display. The picture-sceneclassification display and reproduction-position display unit 19 is alsoproperly referred to hereafter as a classification display unit.

It is to be noted that, instead of processing the digital audio signalreceived from the audio A/D converter 1, the audio classificationprocess unit 20 can also be designed into a configuration for receivingthe audio data, which is obtained as a result of the band compressionprocess carried out by the audio encoder 2 on the digital analog signalgenerated by the audio A/D converter 1, as indicated by a dashed lineshown in FIG. 1, and carrying out the classification processingdescribed earlier on the audio data.

On the other hand, an analog video signal (a video input) supplied tothe recording/reproduction apparatus 400 implemented by the fourthembodiment is fed to a video A/D converter 3 for converting the analogvideo signal into a digital video signal to be supplied to a videoencoder 4.

The video encoder 4 carries out a band compression process on thedigital video signal by adoption of a predetermined compressiontechnique such as an MPEG2 method. Video data obtained as a result ofthe band compression process carried out on the digital video signal issupplied to a video classification process unit 5, the multiplexingprocess unit 6, an I-picture detection unit 21 and a shrunk-pictureprocess unit 22.

The video classification process unit 5 detects picture characteristicsof I pictures of the video data received from the video encoder 4 andrecognizes picture similarity on the basis of the detectedcharacteristics. In addition, the video classification process unit 5also recognizes similar-picture segments as explained earlier byreferring to FIGS. 28A to 28C and 29A to 29C. The video classificationprocess unit 5 also generates an identification information signal foridentifying the similar-picture segments and supplies the signal to themultiplexing process unit 6 and the classification display unit 19. Thesignal identification information includes position information of thevideo data, and the position information of the video data includesinformation on how many consecutive similar frames are included in eachsimilar-picture segment.

It is to be noted that, instead of receiving the compressed video datafrom the video encoder 4, the video classification process unit 5 mayalso receive the digital video signal output by the video A/D converter3 as indicated by a dashed line shown in FIG. 27 and carries out theprocesses to identify and classify similar pictures of the digital videodata in frame units as described above on the basis of the digital videosignal.

In addition, it is also possible to provide a configuration in which theidentification information signals obtained as results of the processesto classify the video and audio signals can also be generated by thesystem controller 17 on the basis of the classification-process resultsreceived from the video classification process unit 5 and the audioclassification process unit 20 and supplied to the multiplexing processunit 6.

As is the case with the recording/reproduction apparatus 100 implementedby the first embodiment described earlier, the system controller 17 is amicrocomputer including a CPU (Central Processing Unit), a ROM (ReadOnly Memory), a RAM (Random Access Memory), a non-volatile memory and atimer circuit, which are used for controlling the other componentscomposing the recording/reproduction apparatus 500.

In this embodiment, the classification display unit 19 receives theclassification result from the audio classification process unit 20, theidentification information signal from the video classification processunit 5, sequentially displaying, among other information, pictureclassifications.

In addition, the multiplexing process unit 6 generates multiplexed databy typically carrying out a multiplexing process on the audio data, thevideo data and the additional identification information signal receivedfrom the video classification process unit 5, supplying the multiplexeddata to the recording process unit 7.

On top of that, the multiplexed data includes additional information onthe genre of a program, that is, program-genre information indicatingthat a program contained in the multiplexed data is a news program, asport program or another kind of program. The program-genre informationcan be extracted from an EPG data included in the digital broadcastsignal or can be entered by the user.

The recording process unit 7 carries out processing including a processto add error correction codes to the multiplexed data and a process tointerleave the data to create a signal to be recorded, recording thesignal into a predetermined area in the recording medium 8.

Then, in the case of this embodiment, as described before, by using theidentification information signal multiplexed in the multiplexed data,it is possible to identify similar-picture segments to be used assummary information. Thus, it is possible to carry out a digestreproduction process by adoption of a predetermined digest reproductionmethod with a predetermined timing after the end of the recordingprocess.

It is to be noted that, much like the first embodiment explained earlierby referring to FIGS. 4 to 6, the video classification process unit 5detects information on characteristics of video data for each I picture,which is used as a processing unit, detects similar pictures on thebasis of the detected information on characteristics and determinessimilar-picture segments each including pictures similar to each other.Also as described earlier, the same identification is assigned tosimilar pictures so that, by using such information on identification,it is possible to identify which range of pictures is included in eachsimilar-picture segment.

In addition, much like the first embodiment explained earlier byreferring to FIG. 7, the audio classification process unit 20 detectssimilarities of sounds for each predetermined time unit and, much likethe second and third embodiments, the audio classification process unit20 also detects audio levels.

<Reproduction System>

Next, the reproduction system is explained. A control signal generatedby the system controller 17 controls the reproduction control unit 9 toread out multiplexed data specified by the user from the recordingmedium 8 and supply the data to the reproduction process unit 10. Thereproduction process unit 10 carries out predetermined processesincluding deinterleave processing on the multiplexed data receivedthereby and supplies the processed multiplexed data to the separationprocess unit 11. The separation process unit 11 demultiplexes themultiplexed data supplied thereto to generate audio data, video data,shrunk-picture data and an identification information signal.

The audio data generated by the separation process unit 11 is suppliedto an audio decoder 12. The audio data supplied to the audio decoder 12is data compressed by adoption of a predetermined compression method.The audio decoder 12 thus decompresses the compressed audio datareceived thereby and supplies audio data obtained as a result of thedecompression to an audio digital/analog converter 13, which isabbreviated hereafter to an audio D/A converter.

The audio D/A converter 13 converts the digital audio signal suppliedthereto into an analog audio signal and outputs the analog audio signalto a speaker. The speaker generates a sound according to the analogaudio signal supplied thereto.

In addition, the video data generated by the separation process unit 11is supplied to a video decoder 14. The video data supplied to the videodecoder 14 is data compressed by adoption of a predetermined compressionmethod. The video decoder 14 thus decompresses the compressed video datareceived thereby and supplies video data obtained as a result of thedecompression to a video digital/analog converter 15, which isabbreviated hereafter to a video D/A converter.

The video D/A converter 15 converts the digital video signal suppliedthereto into an analog video signal and outputs the analog video signalto a monitor receiver. The monitor receiver displays a picture accordingto the analog video signal supplied thereto.

In addition, the identification information signal generated by theseparation process unit 11 is supplied to an identification-signaldetection unit 16. The identification-signal detection unit 16 detectsnecessary identification information signal for digest reproduction fromthe identification information signal received thereby, and supplies theidentification information signal to the system controller 17 and ifneeded, to the classification display unit 19.

In addition, the identification-signal detection unit 16 also extracts apredetermined times tamp signal or frame numbers from the identificationinformation signal received thereby and supplies the timestamp signal orthe frame numbers to the system controller 17. By detecting areproduction point of time on the basis of the timestamp signal, areproduction position can also be displayed on the classificationdisplay unit 19.

When a digest reproduction process is requested for a specified program,the genre of the program is identified from program-genre informationadded to the multiplexed signal of the program. Then, the reproductioncontrol unit 9 is operated so as to carry out the digest reproductionprocess in a digest reproduction mode suitable for the identified genre.In the digest reproduction process, picture data of similar-picturesegments identified by information on identifications in accordance withcontrol executed by the reproduction control unit 9 to serve as summaryinformation is read out from the recording medium.

The picture data read out from the recording medium to be used assummary information is output to a speaker and a monitor receiver forcarrying out the digest reproduction, by way of the classificationdisplay unit 19, the separation process unit 11, the audio decoder 12,the audio D/A converter 13, the video decoder 14 and the video D/Aconverter 15 as described earlier.

It is to be noted that, the digest reproduction mode to be used is by nomeans limited to a mode automatically selected in accordance with anautomatically identified genre of a program even if a digestreproduction mode is automatically identified as described above. Theuser can also select a digest reproduction mode to be used for carryingout a requested digest reproduction process and make a request for useof the selected digest reproduction mode. As another alternative, adigest reproduction mode can be set in advance as a mode to be used forcarrying out a requested digest reproduction process.

<Operations of the Recording/Reproduction Apparatus 500 Implemented bythe Fifth Embodiment>

Next, operations carried out by the recording/reproduction apparatus 500implemented by the fifth embodiment described above in a recordingprocess are explained by referring to a flowchart shown in FIGS. 30 to32. It is to be noted that, in order to make the explanation simple, thefollowing description assumes that the digest reproduction mode A or Bexplained earlier by referring to FIGS. 28A to 28C is used as a digestreproduction mode.

In addition, in this embodiment, as a default special reproduction mode,the digest reproduction mode A or B explained earlier by referring toFIGS. 28A to 28C can be selected in accordance with the program genre.

The processing represented by the flowchart shown in FIGS. 30 to 32 iscarried out by the recording/reproduction apparatus 500 in accordancewith a command entered by the user via the user input I/F 18 to carryout an ordinary or digest reproduction process.

The flowchart begins with a step S401 at which the system controller 17determines whether a reproduction process requested by the user is anordinary or special reproduction process when the user enters a commandmaking a request for the reproduction process. The ordinary reproductionprocess is a process to reproduce all principal information from therecording medium 8 whereas the special reproduction process in thisembodiment is the digest reproduction process.

If the determination result produced by the operation carried out at thestep S401 indicates that the command entered by the user is a commandmaking a request for an ordinary reproduction process, the flow of theoperations goes on to a step S402 at which the system controller 17controls the components of the reproduction control unit 9 associated tothis process, to carry out an ordinary reproduction process ofreproducing all principal information of a program from the recordingmedium 8.

Then, the flow of the operations goes on to a step S403 to determinewhether or not a command to stop the ordinary reproduction process hasbeen received through the user input I/F 18. If the determination resultindicates that no command to stop the ordinary reproduction process hasbeen received, the flow of the operations goes back to the step S401 torepeat the operations described above all over again from this step. Ifthe determination result produced by the operation carried out at thestep S403 indicates that a command to stop the ordinary reproductionprocess has been received, on the other hand, the execution of theoperations represented by the flowchart shown in FIGS. 30 and 32 isended. Then, the recording/reproduction apparatus 500 typically enters astate of waiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S401 indicates that the command entered by the user is a commandmaking a request for a special reproduction process, that is, a digestreproduction process, on the other hand, the flow of the operations goeson to a step S404 at which the system controller 17 examines typicallysetting information stored in its own non-volatile memory in order todetermine whether or not setting of a default reproduction mode isadopted to set a default mode as a digest reproduction mode.

If the default mode setting is not adopted to set a default mode for adigest reproduction mode, in this embodiment, the digest reproductionmode A is assumed. It is needless to say that the default reproductionmode for the digest reproduction mode can be changed to the digestreproduction mode C or D explained earlier by referring to FIGS. 29A to29C.

If the determination result produced by the operation carried out at thestep S404 indicates that setting of a default reproduction mode isadopted, the flow of the operations goes on to a step S405 at whichinformation on the genre of the program to be subjected to the digestreproduction process is acquired. As described before, the informationon the genre of the program has been extracted from an EPG data or thelike and added to the multiplexed data in the recording process. Then,the flow of the operations goes on to a step S406 to determine whetheror not the program to be subjected to the digest reproduction process isa news program.

If the determination result produced by the operation carried out at thestep S406 indicates that the program to be subjected to the digestreproduction process is a news program or if the determination resultproduced by the operation carried out at the step S404 indicates thatthe default mode setting is not adopted, the digest reproduction mode Ais used.

In case the news program is indicated at the step S406, the flow of theoperations goes on to a step S407 at which the system controller 17controls the identification-signal detection unit 16 to detect a framenumber added to the program to be subjected to the digest reproductionprocess. Then, at the next step S408, identification information addedas information for identifying a similar-picture segment is detected.

Subsequently, the flow of the operations goes on to a step S409 todetermine whether or not a video signal segment recognized as asimilar-picture segment on the basis of the detected identificationinformation is a scene of a news caster. If the similar-picture segmentis a scene of a news caster, for example, the similar-picture segmentalso conceivably resembles a news-caster scene of anothersimilar-picture segment. Thus, in the operation carried out at the stepS409, a similar-picture segment resembling another similar-picturesegment is recognized as a similar-picture segment to be subjected tothe digest reproduction process.

If the determination result produced by the operation carried out at thestep S409 indicates that the similar-picture segment to be subjected tothe digest reproduction process is a scene of a news caster, the flow ofthe operations goes on to a step S410 at which frames of thesimilar-picture segments are reproduced. If the determination resultproduced by the operation carried out at the step S409 indicates thatthe similar-picture segment to be subjected to the digest reproductionprocess is not a scene of a news caster, on the other hand, the flow ofthe operations goes on to a step S411 at which frames of thesimilar-picture segments are skipped.

In either case, the flow of the operations then goes on to a step S412to determine whether or not the user has entered a command to stop thedigest reproduction process via the user input I/F 18. If thedetermination result indicates that the user has entered a command tostop the digest reproduction process, the execution of the operationsrepresented by the flowchart shown in FIGS. 30 and 32 is ended. Then,the recording/reproduction apparatus 500 typically enters a state ofwaiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S412 indicates that the user has not entered a command to stop thedigest reproduction process, on the other hand, the flow of theoperations goes on to a step S413 to determine whether or not the digestreproduction process has ended. If the determination result produced bythe operation carried out at the step S413 indicates that the digestreproduction process has ended, the execution of the operationsrepresented by the flowchart shown in FIGS. 30 and 32 is terminated.Then, the recording/reproduction apparatus 500 typically enters a stateof waiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S413 indicates that the digest reproduction process has not ended,on the other hand, the flow of the operations goes on to a step S414 todetermine whether or not the user has entered a command to change thereproduction mode via the user input I/F 18.

If the determination result produced by the operation carried out at thestep S414 indicates that the user has not entered a command to changethe reproduction mode, the flow of the operations goes back to the stepS407 to repeat the operations described above from this step. In thiscase, the digest reproduction process is continued in the digestreproduction mode A.

If the determination result produced by the operation carried out at thestep S414 indicates that the user has entered a command to change thereproduction mode, on the other hand, the flow of the operations goesback to the step S401 to repeat the operations described above all overagain from this step.

If the determination result produced by the operation carried out at thestep S406 indicates that the program to be subjected to the digestreproduction process is not a news program, on the other hand, the flowof the operations goes on to a step S415 of a continuation flowchartshown in FIG. 31. At this step, the digest reproduction process isstarted in the digest reproduction mode B.

At the step S415, the system controller 17 controls theidentification-signal detection unit 16 to detect frame numbers added tothe program to be subjected to the digest reproduction process. Then, atthe next step S416, a variable k is initialized by setting it at 0(zero). The variable k is a counter for counting the number of iterativeexecutions of a loop starting with a step S418.

Subsequently, at the next step S417, the system controller 17 determineswhether of not predetermined identification information associated witha picture segment has been detected. The picture segment indicated bythe identification information is one of picture segments to besequentially reproduced. This picture segment includes a similar-picturesegment and predetermined picture segments immediately preceding andsucceeding the similar-picture segment as explained earlier by referringto FIG. 28C.

If the determination result produced by the operation carried out at thestep S417 indicates that the predetermined identification informationassociated with a picture segment has been detected, the flow of theoperations goes on to the step S418. At this step, a flag, whichindicates that the predetermined identification assigned to frames ofthe similar-picture segment has been detected, is set at 1 for thepurpose of indicating that the digest reproduction operation of thissimilar-picture segment has been completed as will be described later.

Then, the flow of the operations goes on to a step S419 at which thesystem controller 17 determines whether or not operations starting witha step S420 are to be carried out for the first time. Since the variablek has a value of 0, the determination result produced by the operationcarried out at the step S419 indicates that the operations starting witha step S420 have not been carried out so far. In this case, the flow ofthe operations goes on to a step S420 at which a first-frame setvariable n is initialized by setting the variable n at 0 for the purposeof reproducing frames (N−a) to (N−1) immediately preceding frame Nlocated at the head of the similar-picture segment.

Then, at the next step S421, a first one of frames (N−a) to (N−1)immediately preceding frame N located at the head of the similar-picturesegment is reproduced. Subsequently, at the next step S422, the variablen is incremented by 1. Then, at the next step S423, the variable k isincremented by 1. Subsequently, the flow of the operations goes on to astep S424 to determine whether or not frames (N−a) to (N−1) have allbeen reproduced.

If the determination result produced by the operation carried out at thestep S424 indicates that frames (N−a) to (N−1) have not all beenreproduced, the flow of the operations goes back to the step S421 torepeat the operations described above starting from this step. If thedetermination result produced by the operation carried out at the stepS424 indicates that frames (N−a) to (N−1) have all been reproduced, onthe other hand, the flow of the operations goes on to a step S425 atwhich frames of the similar-picture segment are reproduced. The flow ofthe operations also goes on to the step S425 if the determination resultproduced by the operation carried out at the step S419 indicates thatthe predetermined picture segment immediately preceding thesimilar-picture segment has been reproduced.

Subsequently, at the next step S426, the system controller 17 determineswhether or not the user has entered a command to stop the digestreproduction process via the user input I/F 18. At step S426, if thedetermination result indicates that the user has entered a command tostop the digest reproduction process, the execution of the operationsrepresented by the flowchart shown in FIGS. 30 and 32 is ended. Then,the recording/reproduction apparatus 500 typically enters a state ofwaiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S426 indicates that the user has not entered a command to stop thedigest reproduction process, on the other hand, the flow of theoperations goes on to a step S427 to determine whether or not the digestreproduction process to reproduce the desired program has beencompleted. If the determination result produced by the operation carriedout at the step S427 indicates that the digest reproduction process toreproduce the desired program has been completed, the execution of theoperations represented by the flowchart shown in FIGS. 30 and 32 isended. Then, the recording/reproduction apparatus 500 typically enters astate of waiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S427 indicates that the digest reproduction process to reproducethe desired program has not been completed, on the other hand, the flowof the operations goes on to a step S428 to determine whether or not theuser has entered a command to change the reproduction mode. If thedetermination result produced by the operation carried out at the stepS428 indicates that the user has not entered a command to change thereproduction mode, the flow of the operations goes back to the step S417to repeat the operations described above from this step. In this case,the digest reproduction process is continued in the digest reproductionmode B.

If the determination result produced by the operation carried out at thestep S428 indicates that the user has entered a command to change thereproduction mode, on the other hand, the flow of the operations goesback to the step S401 of the main flowchart shown in FIG. 30 to repeatthe operations described above all over again from this step.

If the determination result produced by the operation carried out at thestep S417 indicates that the predetermined identification information ofa similar-picture segment to be reproduced was not detected to provethat the reproduction of the similar-picture segment has been completed,the flow of the operations goes on to a step S429 of anothercontinuation flowchart shown in FIG. 32. At this step, the digestreproduction process of a predetermined picture segment immediatelyfollowing the reproduced similar-picture segment is started.

First of all, at the step S429, the system controller 17 examines theflag, which is set at the step S418, to determine whether the flag hasbeen indeed set at 1, that is, whether or not the digest reproductionprocess to reproduce the segment immediately preceding thesimilar-picture segment and the similar-picture segment itself has beencompleted.

If the determination result produced by the operation carried out at thestep S429 indicates that the flag has been set at 1, the flow of theoperations goes on to a step S430 at which the flag is reset to 0. Then,at the next step S431, the variable n is initialized at 0. Subsequently,at the next step S432, frame (N+n) immediately following thesimilar-picture segment is reproduced. Then, at the next step S433, thevariable n is incremented by 1.

Subsequently, the flow of the operations goes on to a step S434 todetermine whether or not the variable n has exceeded a predeterminedvalue b, that is, whether or not the digest reproduction process toreproduce a predetermined segment immediately following thesimilar-picture segment has been completed. If the determination resultproduced by the operation carried out at the step S434 indicates thatthe digest reproduction process to reproduce the predetermined segmenthas not been completed, the flow of the operations goes back to the stepS432 to repeat the operations described above from this step.

If the determination result produced by the operation carried out at thestep S434 indicates that the digest reproduction process to reproducethe predetermined segment immediately following the similar-picturesegment has been completed, on the other hand, the flow of theoperations goes on to a step S435 at which the system controller 17determines whether or not the user has entered a command to stop thedigest reproduction process via the user input I/F 18. If thedetermination result indicates that the user has entered a command tostop the digest reproduction process, the execution of the operationsrepresented by the flowchart shown in FIGS. 30 and 32 is ended. Then,the recording/reproduction apparatus 500 typically enters a state ofwaiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S435 indicates that the user has not entered a command to stop thedigest reproduction process, on the other hand, the flow of theoperations goes on to a step S436 to determine whether or not the digestreproduction process to reproduce the desired program has beencompleted. If the determination result produced by the operation carriedout at the step S436 indicates that the digest reproduction process toreproduce the desired program has been completed, the execution of theoperations represented by the flowchart shown in FIGS. 30 and 32 isended. Then, the recording/reproduction apparatus 500 typically enters astate of waiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S436 indicates that the digest reproduction process to reproducethe desired program has not been completed, on the other hand, the flowof the operations goes on to a step S437 to determine whether or not theuser has entered a command to change the reproduction mode. If thedetermination result produced by the operation carried out at the stepS437 indicates that the user has not entered a command to change thereproduction mode, the flow of the operations goes back to the step S417of the continuation flowchart shown in FIG. 31 to repeat the operationsdescribed above from this step. In this case, the digest reproductionprocess is continued in the digest reproduction mode B.

If the determination result produced by the operation carried out at thestep S437 indicates that the user has entered a command to change thereproduction mode, on the other hand, the flow of the operations goesback to the step S401 of the main flowchart shown in FIG. 30 to repeatthe operations described above all over again from this step.

It is to be noted that, if the determination result produced by theoperation carried out at the step S429 indicates that the flag has notbeen set at 1, on the other hand, the flow of the operations goes on toa step S438 at which the flag is reset to 0. Then, at the next stepS439, frames being processed are skipped. Subsequently, the flow of theoperations goes on to the step S435 as shown in FIG. 32.

It is also worth noting that, even though the above description explainsthe use of the digest reproduction modes A and B, the embodiment is notlimited to these modes. For example, the above description is alsoapplicable to the digest reproduction modes C and D.

In addition, if the information on the genre of a program indicates thatthe program is a sport program, the digest reproduction mode B or D canalso be used.

<Summary of the Fifth Embodiment>

As described above, in a digest reproduction process to reproducesegments of a program from a recording medium, therecording/reproduction apparatus implemented by the fifth embodimentuses a digest reproduction mode selected thereby in accordance with thegenre of the program or in accordance with information entered by theuser. Thus, the digest reproduction process can be carried outeffectively and with a high degree of efficiency.

In addition, even if a plurality of digest reproduction modes can beused to carry out a digest reproduction process effectively and with ahigh degree of efficiency, with a simple configuration of therecording/reproduction apparatus, a digest reproduction process can becarried out effectively and with a high degree of efficiency withoutcausing the configuration to become complex.

It is to be noted that the digest reproduction mode is by no meanslimited to the four digest reproduction modes A, B, C and D explainedearlier by referring to FIGS. 28A to 28C and 29A to 29C. That is to say,any other digest reproduction modes having a variety of reproductionpatterns can also be used.

In addition, in the recording/reproduction apparatus 500 implemented bythe fifth embodiment, similarity of pictures and similar-picturesegments are recognized in I-picture units. However, the recognitionunit is by no means limited to the I picture. For example, similarity ofpictures and similar-picture segments can also be recognized in frameunits. As another alternative, similarity of pictures andsimilar-picture segments are recognized in I-picture units whileprocessing of others is carried out in frame units.

On top of that, in the recording/reproduction apparatus 500 implementedby the fifth embodiment, video information is classified intosimilar-picture segments on the basis of information on characteristicsof the video information. However, the basis used in classifying thevideo information into similar-picture segments is by no means limitedto characteristics of the video information. For example,similar-picture segments usable as summary information can also be seton the basis of an audio signal. As another alternative, similar-picturesegments usable as summary information can also be set on the basis ofboth audio and video signals.

Sixth Embodiment

Even in the case of the fifth embodiment described above, if each ofsimilar-picture segments included in a video signal representing atelevision broadcast program stored on the recording medium becomes toolong, contents of the television broadcast program cannot be graspedfast even by carrying out a digest reproduction process to produce thesimilar-picture segments. In order to solve this problem, only apredetermined portion of every similar-picture segment is reproduced asshown in FIGS. 29B and 29C.

Nevertheless, there is also conceivably a television broadcast programincluding a very long continuous similar-picture segment. Examples ofsuch a television broadcast program are a news-analysis program or adiscussion program. In a news-analysis program or a discussion program,a scene showing a speaking performer continues over a long period oftime. Another example is a marathon program or a relay-road raceprogram. In such programs, a picture taken from an unchanged cameraangle as a picture showing the running condition of a runner maycontinue over a long period of time.

In such a case, if a digest reproduction process is carried out toreproduce only a predetermined portion at the head of everysimilar-picture segment, it may be conceivably impossible to effectivelyand efficiently grasp contents of a program including thesimilar-picture segments reproduced in the digest reproduction process.

In order to solve this problem, in a recording/reproduction apparatusimplemented by a sixth embodiment, with a simple configuration of therecording/reproduction apparatus, a digest reproduction process can becarried out effectively and efficiently without causing theconfiguration to become complex even if a similar-picture segmentcontinues over a relatively long period of time.

It is to be noted that the recording/reproduction apparatus implementedby the sixth embodiment has the same configuration as therecording/reproduction apparatus 500 implemented by the fifth embodimentexplained earlier by referring to FIG. 27. Thus, therecording/reproduction apparatus implemented by the sixth embodiment isexplained by also referring to FIG. 27 showing therecording/reproduction apparatus 500 implemented by the fifthembodiment.

Much like the recording/reproduction apparatus 500 implemented by thefifth embodiment, the recording/reproduction apparatus 500 implementedby the sixth embodiment having a configuration shown in FIG. 27 iscapable of data-compressing video and audio signals as digital signalsof a television broadcast program, storing the compressed digital videoand audio signals onto a recording medium and reproducing the compresseddigital video and audio signals from the recording medium.

In a process to store compressed digital video and audio signals onto arecording medium, much like the other embodiments explained earlier,similar I pictures are detected and classified into similar-picturesegments, and identification information is added to make each of thesimilar-picture segments identifiable before storing the video and audiosignals onto the recording medium.

In addition, also in the case of the recording/reproduction apparatus500 implemented by the sixth embodiment, basically, a digestreproduction process is carried out by reproducing similar-picturesegments. In this case, however, the length of every similar-picturesegment is taken into consideration. FIGS. 33A and 33B as well as FIGS.34A to 34C are explanatory diagrams showing the principle of operationof the recording/reproduction apparatus 500 implemented by the sixthembodiment.

In an input picture series shown in FIG. 33A, for example, pictures 1, 2and 3 are similar to each other, pictures 13, 14, 15, 16 and 17 aresimilar to each other and pictures 22, 23, 24, 25, 26, 27, 28 and 29 aresimilar to each other. In addition, a reproduction segment length (or aset segment length) t is set in advance as a length considered to beproper for a digest reproduction process as shown in FIG. 33A.

In accordance with this set segment length t, segments A1, A2 and A3 aretaken as picture segment for a digest reproduction process as shown inFIG. 33A. If a similar-picture segment has a length greater than the setsegment length t, a portion of the similar-picture segment beyond theset segment length t is examined. If the portion is longer than apredetermined interval segment length s, a part of the portion beyondthe predetermined interval segment length s is taken as anotheradditional picture segment. This other additional picture segment isalso trimmed if necessary to a length not exceeding the set segmentlength t.

That is to say, a segment A4 shown in FIG. 33A is the other additionalpicture segment for a digest reproduction process. In the case of thetypical input picture series shown in FIG. 33A, video signalsrepresenting the picture segments A1, A2, A3 and A4 extracted from theinput picture series shown in FIG. 33A are used in a digest reproductionprocess as shown in FIG. 33B.

Thus, in the case of a similar-picture segment having a length exceedingthe sum of the set segment length t and the predetermined intervalsegment length s, a plurality of picture segments used for a digestreproduction process is set. In this way, even if a program includes along similar-picture segment, contents of the program subjected to adigest reproduction process can be grasped effectively and efficiently.

In addition, besides similar-picture segments, levels of the audiosignal are also taken into consideration in setting picture segmentsused for a digest reproduction process as is the case with an exampleshown in FIGS. 34A to 34C. FIG. 34A is a diagram showing an inputpicture series and FIG. 34B is a diagram showing an input sound seriesaccompanying the input picture series shown FIG. 34A.

In an input picture series shown in FIG. 34A, for example, pictures 1, 2and 3 are similar to each other, pictures 13, 14, 15, 16, 17 and 18 aresimilar to each other and pictures 22, 23, 24, 25, 26, 27, 28 and 29 aresimilar to each other. In addition, much like the input signal seriesshown in FIG. 33A, a set segment length t and a predetermined intervalsegment length s are used in setting picture segments to be utilized fora digest reproduction process. In the case of the input picture seriesshown in FIG. 34A, however, a segment with an audio signal exceeding apredetermined level is preferred as a picture segment used for a digestreproduction process.

Thus, in the case of the input picture series shown in FIG. 34A, asdescribed above, pictures 13, 14, 15, 16, 17 and 18 are similar to eachother. However, the level of an audio signal for pictures 16, 17 and 18is higher as shown in FIG. 34B. Accordingly, as a picture segment usedfor a digest reproduction process, a segment comprising pictures 16, 17and 18 is set as shown in FIG. 34A.

As described above, a portion included in a similar-picture segment as aportion having audio signal exceeding a predetermined level and having alength not exceeding the set segment length t is set as a picturesegment used for a digest reproduction process. In addition, if asimilar-picture segment is long so that its portion set as a picturesegment used for a digest reproduction process has a length greater thanthe set segment length t, the portion beyond the set segment length t isexamined. If the portion is longer than a predetermined interval segmentlength s, a portion part beyond the predetermined interval segmentlength s is taken as another additional picture segment. This otheradditional picture segment is also trimmed if necessary to a length notexceeding the set segment length t.

By doing so, in the case of an input picture series shown in FIG. 34A,segments B1, B2, B3 and B4 are each set as a picture segment used for adigest reproduction process. The digest reproduction process is thencarried out to reproduce the segments B1, B2, B3 and B4 consecutively asshown in FIG. 34C.

It is to be noted that, in the case of a long similar-picture segmentlike the ones shown in FIGS. 33A and 34A, a plurality of picturesegments used for a digest reproduction process is set and a skipsegment having a length equal to the predetermined interval segmentlength s is resulted in within the similar-picture segment.

<Operations and Configuration of the Recording/Reproduction Apparatus500 Implemented by the Sixth Embodiment>

Operations and configuration of the recording/reproduction apparatus 500implemented by the fifth embodiment shown in FIG. 27 are explained.Operations carried out by a recording system of the sixth embodiment arethe same as those in the fifth embodiment explained before. If the levelof an audio signal is taken into consideration as explained earlier byreferring to FIGS. 34A to 34C, however, the audio classification processunit 20 supplies identification information for identifying anaudio-signal segment having an audio-signal level at least equal to apredetermined level to the multiplexing unit 6. The multiplexing unit 6also adds this identification information to multiplexed data so that apicture segment in a similar-picture segment can be recognized as apicture segment corresponding to an audio-signal segment having anaudio-signal level at least equal to the predetermined level.

In addition, operations carried out by a reproduction system of thesixth embodiment are also the same as those in the fifth embodimentexplained before. Furthermore, when a command making a request forexecution of a digest reproduction process is received, in order tocarry out the digest reproduction process, a signal of picture segmentsfor the digest reproduction process is read out from the recordingmedium 8 in accordance with control executed by the reproduction controlunit 9, which is controlled by the system controller 17 as a part ofmultiplexed data. The multiplexed data has been recorded on therecording medium 8 along with identification information added to themultiplexed data as information for identifying similar-picture segmentsincluded in the multiplexed data as a result of classificationprocessing. In this case, if a similar-picture segment has a lengthgreater than the set segment length t, processing to skip a portion withduration equal to the interval segment length s is carried out.

On top of that, signals including a predetermined time stamp signaldetected by the identification-signal detection unit 16 are supplied tothe system controller 17, which then detects a reproduction point oftime from these signals in order to display the present reproductionposition on the classification display unit 19 and display a picturescene at a skipped point as a representative picture on theclassification display unit 19 or the like. In this way, it is possibleto make a digest reproduction process easily understandable to the user.

<Operations of the Recording/Reproduction Apparatus 500 Implemented bythe Sixth Embodiment>

By referring to a flowchart shown in FIGS. 35 and 36, the followingdescription explains operations carried out by therecording/reproduction apparatus 500 of the sixth embodiment duringreproduction and recording processes. The operations represented by theflowchart shown in FIGS. 35 and 36 are processing carried out by therecording/reproduction apparatus 500 when the user enters a commandmaking a request for execution of a reproduction or recording processvia the user input I/F 18.

It is to be noted that, in the case of a requested reproduction process,in the following description, it is assumed that multiplexed data hasalready been stored on the recording medium 8 along with the sameidentification information for every similar-picture segment obtained asa result of similarity classification processing.

When the user enters a command making a request for a reproduction orrecording process, the flowchart begins with a step S501 at which thesystem controller 17 determines whether the command is a commandrequesting a reproduction or recording process. If the determinationresult produced by the operation carried out at the step S501 indicatesthat the command is a command making a request for a reproductionprocess, the flow of the operations goes on to a step S502 at which thesystem controller 17 controls components of the recording/reproductionapparatus 500 to put the recording/reproduction apparatus 500 in areproduction mode to start an operation to read out multiplexed data ofa desired program from the recording medium 8.

Then, at the next step S503, the reproduction command is furtherexamined to determine whether or not the requested reproduction processis a special, that is a digest reproduction process. If thedetermination result produced by the operation carried out at the stepS503 indicates that the requested reproduction process is a digestreproduction process, the flow of the operations goes on to a step S504at which identification information of the multiplexed data to bereproduced through the identification-signal detection unit 16 isdetected. Thereby, the flow of the operations goes on to a step S505 todetermine whether or not the information signal to be reproduced ismultiplexed data of a picture segment for a digest reproduction process.

If the determination result produced by the operation carried out at thestep S505 indicates that the information signal to be reproduced ismultiplexed data of a picture segment for a digest reproduction process,the flow of the operations goes on to a step S506 at which the digestreproduction process is carried out by reproducing the multiplexed data,and then, at the next step S507, the length of a lapsing reproductiontime is detected.

Subsequently, the flow of the operations goes on to a step S508 todetermine whether or not a length detected at the step S507 as thelength of the lapsing reproduction time has become equal to the setsegment length t. If the determination result produced by the operationcarried out at the step S508 indicates that data of the length equal tothe set segment length t has been reproduced, the flow of the operationsgoes on to a step S509 at which an operation to skip the intervalsegment length s is carried out. Subsequently, the flow of theoperations goes on to a step S510 to determine whether or not the userhas entered a command to stop the digest reproduction process.

If the determination result produced by the operation carried out at thestep S510 indicates that the user has not entered a command to stop thedigest reproduction process, the flow of the operations goes on to astep S511 to determine whether or not the user has entered a command tochange the operation mode. If the determination result produced by theoperation carried out at the step S511 indicates that the user has notentered a command to change the operation mode, the flow of theoperations goes back to the step S504 to repeat the operations describedabove from this step. In this way, the digest reproduction process iscontinued.

If the determination result produced by the operation carried out at thestep S511 indicates that the user has entered a command to change theoperation mode, on the other hand, the flow of the operations goes backto the step S501 to repeat the operations described above all over againfrom this step. In this way, it is possible to carry out an ordinaryreproduction process to reproduce a program from the recording medium, adigest reproduction process to reproduce a program from the recordingmedium or a recording process to store a program onto the recordingmedium in accordance with a command entered by the user.

If the determination result produced by the operation carried out at thestep S510 indicates that the user has entered a command to stop thedigest reproduction process, on the other hand, the system controller 17ends the execution of the operations represented by the flowchart shownin FIGS. 35 and 36. Then, the recording/reproduction apparatus 500typically enters a state of waiting for the user to enter a nextcommand.

If the determination result produced by the operation carried out at thestep S508 indicates that the digest reproduction process to reproducedata of the length equal to the set segment length t has not beencompleted, on the other hand the flow of the operations goes on to astep S512 of a continuation flowchart shown in FIG. 36 to determinewhether or not the user has entered a command to stop the digestreproduction process.

If the determination result produced by the operation carried out at thestep S512 indicates that the user has not entered a command to stop thedigest reproduction process, the flow of the operations goes on to astep S513 to determine whether or not the user has entered a command tochange the operation mode. If the determination result produced by theoperation carried out at the step S513 indicates that the user has notentered a command to change the operation mode, the flow of theoperations goes back to the step S504 of the main flowchart shown inFIG. 35 to repeat the operations described above from this step. In thisway, the digest reproduction process is continued.

If the determination result produced by the operation carried out at thestep S513 indicates that the user has entered a command to change theoperation mode, on the other hand, the flow of the operations goes backto the step S501 of the main flowchart shown in FIG. 35 to repeat theoperations described above all over again from this step. In this way,it is possible to carry out an ordinary reproduction process toreproduce a program from the recording medium, a digest reproductionprocess to reproduce a program from the recording medium or a recordingprocess to store a program onto the recording medium in accordance witha command entered by the user.

If the determination result produced by the operation carried out at thestep S512 indicates that the user has entered a command to stop thedigest reproduction process, on the other hand, the system controller 17ends the execution of the operations represented by the flowchart shownin FIGS. 35 and 36. Then, the recording/reproduction apparatus 500typically enters a state of waiting for the user to enter a nextcommand.

If the determination result produced by the operation carried out at thestep S501 indicates that the command is a command making a request for arecording process, on the other hand, the flow of the operations goes onto a step S514 of the continuation flowchart shown in FIG. 36. At thestep S514, the system controller 17 controls components of therecording/reproduction apparatus 500 to put the recording/reproductionapparatus 500 in a recording mode to start an operation to storesupplied video and audio signals onto the recording medium 8.

Then, the flow of the operations goes on to a step S515 to determinewhether or not the user has entered a command to stop the recordingprocess. If the determination result produced by the operation carriedout at the step S515 indicates that the user has not entered a commandto stop the recording process, the flow of the operations goes on to astep S516 to determine whether or not the user has entered a command tochange the operation mode.

If the determination result produced by the operation carried out at thestep S516 indicates that the user has not entered a command to changethe operation mode, the flow of the operations goes back to the stepS514 to repeat the operations described above from this step. In thisway, the recording process is continued.

If the determination result produced by the operation carried out at thestep S516 indicates that the user has entered a command to change theoperation mode, on the other hand, the flow of the operations goes backto the step S501 of the main flowchart shown in FIG. 35 to repeat theoperations described above all over again from this step. In this way,it is possible to carry out an ordinary reproduction process toreproduce a program from the recording medium, a digest reproductionprocess to reproduce a program from the recording medium or a recordingprocess to store a program onto the recording medium in accordance witha command entered by the user.

If the determination result produced by the operation carried out at thestep S514 indicates that the user has entered a command to stop therecording process, on the other hand, the system controller 17 ends theexecution of the operations represented by the flowchart shown in FIGS.35 and 36. Then, the recording/reproduction apparatus 500 typicallyenters a state of waiting for the user to enter a next command.

If the determination result produced by the operation carried out at thestep S503 of the main flowchart shown in FIG. 35 indicates that therequested reproduction process is not a digest reproduction process, onthe other hand, the flow of the operations goes on to a step S517 of thecontinuation flowchart shown in FIG. 36. At this step, the systemcontroller 17 controls components of the recording/reproductionapparatus 500 to put the recording/reproduction apparatus 500 in anordinary reproduction mode to start an ordinary reproduction process toreproduce multiplexed data of a desired program from the recordingmedium 8.

Subsequently, the flow of the operations goes on to a step S518 todetermine whether or not the user has entered a command to stop theordinary reproduction process. If the determination result produced bythe operation carried out at the step S518 indicates that the user hasnot entered a command to stop the ordinary reproduction process, theflow of the operations goes on to a step S519 to determine whether ornot the user has entered a command to change the operation mode.

If the determination result produced by the operation carried out at thestep S519 indicates that the user has not entered a command to changethe operation mode, the flow of the operations goes back to the stepS517 to repeat the operations described above from this step. In thisway, the digest reproduction process is continued.

If the determination result produced by the operation carried out at thestep S519 indicates that the user has entered a command to change theoperation mode, on the other hand, the flow of the operations goes backto the step S501 of the main flowchart shown in FIG. 35 to repeat theoperations described above all over again from this step. In this way,it is possible to carry out an ordinary reproduction process toreproduce a program from the recording medium, a digest reproductionprocess to reproduce a program from the recording medium or a recordingprocess to store a program onto the recording medium in accordance witha command entered by the user.

If the determination result produced by the operation carried out at thestep S517 indicates that the user has entered a command to stop theordinary reproduction process, on the other hand, the system controller17 ends the execution of the operations represented by the flowchartshown in FIGS. 35 and 36. Then, the recording/reproduction apparatus 500typically enters a state of waiting for the user to enter a nextcommand.

<Summary of the Sixth Embodiment>

As described above, in the recording/reproduction apparatus implementedby the sixth embodiment, even if a relatively long similar-picturesegment exists in a recorded program, a digest reproduction process canbe carried out by properly setting a picture segment for the digestreproduction process so as that the user can grasp contents of theprogram effectively and efficiently.

It is to be noted that the user is of course allowed to determine theset segment length t and the interval segment length s in advance orwith a proper timing.

<Others>

In the first to sixth embodiments, the picture similarity processing andthe processing to create similar-picture segments are based on I-pictureunits of compressed MPEG data. However, the basis of these pieces ofprocessing is by no means limited to I pictures of compressed MPEG data.For example, characteristics can also be detected for each frame ofpre-compression data and similar-picture segments are then generated onthe basis of the detected characteristics.

In addition, it is also possible to provide a recording/reproductionapparatus with a configuration including all functions of therecording/reproduction apparatus implemented by the first to sixthembodiments or a configuration including functions of only properlyselected two or more recording/reproduction apparatus implemented by thefirst to sixth embodiments.

On top of that, the recording medium 8 of the recording/reproductionapparatus is by no means limited to a magnetic disk such as a hard disk,but the recording medium 8 can be any kind of randomaccessible-recording medium having a relatively large storage capacitysuch as an optical disk, a magneto-optical disk or a semiconductormemory.

Furthermore, apparatus implemented by the embodiments described aboveare by no means limited to recording/reproduction apparatus, but theapparatus can also be, among others, a network-type home serverconnected to various kinds of home electronic equipment such as acommunication apparatus and a broadcast-use receiving apparatus.

Moreover, the present invention can also be used as a method forcontrolling reproduction of information such as video and audioinformation by using data including the aforementioned shrunk pictures(or thumb-nail pictures), the summary information cited above and theaforementioned identifications for associating the shrunk pictures andthe summary information with principal information as the so-called metadata.

INDUSTRIAL APPLICABILITY

A digital information signal recorded on a recording medium having alarge storage capacity can be utilized effectively and efficiently, andefficient utilization of the recording medium can also be promoted. Inaddition, it is possible to provide an information-signal processapparatus offering much convenience to the user.

1. An information-signal process apparatus comprising: characteristicinformation detection means for detecting information on characteristicsof a first information signal to be recorded for each predeterminedprocessing unit of said first information signal; identificationinformation generation means for generating identification informationfor identifying a predetermined signal segment of said first informationsignal on the basis of a detection result produced by saidcharacteristic information detection means; and recording means forrecording said first information signal in a first recording area of arecording medium and a second information signal in a second recordingarea of said recording medium wherein: said first and second recordingareas are set in accordance with a recording method determined inadvance for said recording medium; and said second information signal isan information signal included in said first information signal as aninformation signal in said predetermined signal segment identified bysaid identification information, wherein said information-signal processapparatus further having selection-input reception means for receiving aselection input specifying a selected recording method, wherein saidrecording means: sets said first and second recording areas in saidrecording medium in accordance with a selected recording methodspecified by a selection input received by said selection-inputreception means; and stores a first information signal in said firstrecording area and a second information signal in said second recordingarea.
 2. The information-signal process apparatus according to claim 1wherein said recording means records a plurality of said firstinformation signals in said first recording area having a variablestorage size for accommodating said first information signals as acluster of said first information signals and a plurality of said secondinformation signals in said second recording area having a variablestorage size for accommodating said second information signals as acluster of said second information signals.
 3. The information-signalprocess apparatus according to claim 1 wherein said recording meansprovides said first recording area for recording only one piece of saidfirst information signal and said second recording area for recordingonly one piece of said second information signal, and records said pieceof said first information signal and said piece of said secondinformation signal in said first and second recording areas,respectively.
 4. The information-signal process apparatus according toclaim 1 wherein said selected recording method can be at least: a firstrecording method of recording a plurality of said first informationsignals in said first recording area having a variable storage size foraccommodating said first information signals as a cluster of said firstinformation signals and a plurality of said second information signalsin said second recording area having a variable storage size foraccommodating said second information signals as a cluster of saidsecond information signals; or a second recording method of providingsaid first recording area for recording only one piece of said firstinformation signal and said second recording area adjacent to said firstrecording area as an area for recording only one piece of said secondinformation signal to form an alternating arrangement, and recordingsaid piece of said first information signal and said piece of saidsecond information signal in said first and second recording areas,respectively.
 5. An information-signal process apparatus comprising:characteristic information detection means for detecting information oncharacteristics of a first information signal to be recorded for eachpredetermined processing unit of said first information signal;identification information generation means for generatingidentification information for identifying a predetermined signalsegment of said first information signal on the basis of a detectionresult produced by said characteristic information detection means; andrecording means for recording said first information signal in a firstrecording area of a recording medium and a second information signal ina second recording area of said recording medium wherein: said first andsecond recording areas are set in accordance with a recording methoddetermined in advance for said recording medium; and said secondinformation signal is an information signal included in said firstinformation signal as an information signal in said predetermined signalsegment identified by said identification information; storage-sizedetection means for detecting the storage size of a free area left insaid recording medium; and recording control means for executing controlto record a most recent first information signal in a sub-area includedin said first recording area as a sub-area for recording a least recentfirst information over said least recent first information withouteroding said second recording area in case a detection result producedby said storage-size detection means indicates that the storage size ofa free area left in said recording medium is not large enough forstoring said least recent first information signal in a process torecord said first information signal onto said recording medium.
 6. Aninformation-signal process method, executed by a processor, comprisingthe steps of detecting information on characteristics of a firstinformation signal to be recorded for each predetermined processing unitof said first information signal; generating identification informationfor identifying a predetermined information signal of said firstinformation signal on the basis of said detected information oncharacteristics of said first information signal; and recording saidfirst information signal in a first recording area of a recording mediumand a second information signal in a second recording area of saidrecording medium wherein: said first and second recording areas are setin accordance with a recording method determined in advance for saidrecording medium; and said second information signal is saidpredetermined information signal identified by said identificationinformation and located in a predetermined signal segment of said firstinformation signal; receiving a selection input specifying a selectedrecording method; setting said first and second recording areas in saidrecording medium in accordance with a selected recording methodspecified by said selection input specifying said selected recordingmethod; and recording a first information signal in said first recordingarea and a second information signal in said second recording area. 7.The information-signal process method according to claim 6 whereby aplurality of said first information signals is recorded in said firstrecording area having a variable storage size for accommodating saidfirst information signals as a cluster of said first information signalsand a plurality of said second information signals is recorded in saidsecond recording area having a variable storage size for accommodatingsaid second information signals as a cluster of said second informationsignals.
 8. The information-signal process method according to claim 6wherein said first recording area for recording only one piece of saidfirst information signal and said second recording area for recordingonly one piece of said second information signal are provided, and saidpiece of said first information signal and said piece of said secondinformation signal are recorded in said first and second recordingareas, respectively.
 9. The information-signal process method accordingto claim 6, wherein said selected recording method can be at least: afirst recording method of recording a plurality of said firstinformation signals in said first recording area having a variablestorage size for accommodating said first information signals as acluster of said first information signals and a plurality of said secondinformation signals in said second recording area having a variablestorage size for accommodating said second information signals as acluster of said second information signals; or a second recording methodof providing said first recording area for recording only one piece ofsaid first information signal and said second recording area adjacent tosaid first recording area as an area for recording only one piece ofsaid second information signal to form an alternating arrangement andrecording said piece of said first information signal and said piece ofsaid second information signal in said first and second recording areas,respectively.
 10. An information-signal process method, executed by aprocessor, comprising the steps of detecting information oncharacteristics of a first information signal to be recorded for eachpredetermined processing unit of said first information signal;generating identification information for identifying a predeterminedinformation signal of said first information signal on the basis of saiddetected information on characteristics of said first informationsignal; and recording said first information signal in a first recordingarea of a recording medium and a second information signal in a secondrecording area of said recording medium wherein: said first and secondrecording areas are set in accordance with a recording method determinedin advance for said recording medium; said second information signal issaid predetermined information signal identified by said identificationinformation and located in a predetermined signal segment of said firstinformation signal; detecting the storage size of a free area left insaid recording medium; and executing control to record a most recentfirst information signal in a sub-area included in said first recordingarea as a sub-area for recording a least recent first information oversaid least recent first information without eroding said secondrecording area in case a detection result produced by said storage-sizedetection means indicates that the storage size of a free area left insaid recording medium is not large enough for storing said least recentfirst information signal in a process to record said first informationsignal onto said recording medium.
 11. An information-signal processapparatus comprising: characteristic information detection means fordetecting information on characteristics of a first information signalto be recorded for each predetermined processing unit of said firstinformation signal; identification information generation means forgenerating identification information for identifying a predeterminedinformation signal of said first information signal on the basis of adetection result produced by said characteristic information detectionmeans; identification information addition means for adding saididentification information generated by said identification informationgeneration means to said first information signal; first recording meansfor recording said first information signal including saididentification information added thereto by said identificationinformation addition means onto a first recording medium; storage-sizedetection means for detecting the storage size of a free area left insaid first recording medium with a predetermined timing; and secondrecording means for moving all data of one or more said firstinformation signals from said first recording medium to a secondrecording medium and leaving an information signal, which is stored in asignal segment included in each of said moved first information signalsand identified by said identification, in said first recording medium asa second information signal in case a detection result produced by saidstorage-size detection means indicates that the storage size of a freearea left in said recording medium is equal to or smaller than apredetermined value.
 12. An information-signal process apparatuscomprising: characteristic information detection means for detectinginformation on characteristics of a first information signal to berecorded for each predetermined processing unit of said firstinformation signal; identification information generation means forgenerating an identification information for identifying a predeterminedinformation signal of said first information signal on the basis of adetection result produced by said characteristic information detectionmeans; first recording means for recording said first information signalonto a first recording medium along with a second information signal,which is said predetermined information signal identified by saididentification information and included in said first informationsignal; storage-size detection means for detecting the storage size of afree area left in said first recording medium with a predeterminedtiming; and second recording means for moving only one or more saidfirst information signals from said first recording medium to a secondrecording medium in case a detection result produced by saidstorage-size detection means indicates that the storage size of a freearea left in said recording medium is smaller than a predeterminedvalue.
 13. An information-signal process method, executed by aprocessor, comprising the steps of: detecting information oncharacteristics of a first information signal to be recorded for eachpredetermined processing unit of said first information signal;generating identification information for identifying a predeterminedinformation signal of said first information signal on the basis of saiddetected information on characteristics of said first informationsignal; adding said generated identification to said first informationsignal; recording said first information signal including saididentification information added thereto onto a first recording medium;detecting the storage size of a free area left in said first recordingmedium with a predetermined timing; and moving all data of one or moresaid first information signals from said first recording medium to asecond recording medium and leaving an information signal, which isstored in a signal segment included in each of said moved firstinformation signals and identified by said identification information,in said first recording medium as a second information signal in case adetection result indicates that the storage size of a free area left insaid first recording medium is smaller than a predetermined value. 14.An information-signal process method, executed by a processor,comprising the steps of: detecting information on characteristics of afirst information signal to be recorded for each predeterminedprocessing unit of said first information signal; generatingidentification information for identifying a predetermined informationsignal of said first information signal on the basis of said detectedinformation on characteristics of said first information signal;recording said first information signal onto a first recording mediumalong with a second information signal, which is said predeterminedinformation signal identified by said identification information andincluded in said first information signal; detecting the storage size ofa free area left in said first recording medium with a predeterminedtiming; and moving only all or a portion of one or more said firstinformation signals from said first recording medium to a secondrecording medium in case a detection result indicates that the storagesize of a free area left in said first recording medium is smaller thana predetermined value.
 15. An information-signal process apparatus forreproducing an information signal recorded onto a recording medium aftercompleting processes of detecting characteristics of said informationsignal for each predetermined processing unit, classifying segments ofsaid information signal into similar-picture segments exhibitingsimilarity of said characteristics on the basis of said detectedcharacteristics and adding identifications for identifying saidclassified segments to said information signal, said information-signalprocess apparatus comprising: execution-command-input reception meansfor receiving a command input making a request for execution of aspecial reproduction process; read means for reading out saidinformation signal being reproduced from said recording medium; andreproduction control means for executing control to reproduce portionsof each of said classified segments, which are read out by said readmeans from said recording medium as segments of said information signal,from said recording medium upon reception of said command input making arequest for execution of a special reproduction process through saidexecution-command-input reception means, wherein said portionsreproduced from said recording medium each have a length equal to apredetermined set segment length and are separated from each other by apredetermined interval segment length.
 16. The information-signalprocess apparatus according to claim 15, further havingimportant-segment detection means for detecting an important segmentamong segments included in said information signal as said segmentsexhibiting similarity of said characteristics, wherein said reproductioncontrol means sets such a reproduction segment that said reproductionsegment has a length equal to said predetermined set segment length andincludes said important segment detected by said important-segmentdetection means.
 17. An information-signal process method, executed by aprocessor, for reproducing an information signal recorded onto arecording medium after completing processes of detecting characteristicsof said information signal for each predetermined processing unit,classifying segments of said information signal into similar-picturesegments exhibiting similarity of said characteristics on the basis ofsaid detected characteristics and adding identifications for identifyingsaid classified segments to said information signal, saidinformation-signal process method comprising the steps of: receiving acommand input making a request for execution of a special reproductionprocess; reading out said information signal being reproduced from saidrecording medium upon reception of said command input making a requestfor execution of a special reproduction process; and executing controlto reproduce portions of each of said classified segments, which areread out from said recording medium as segments of said informationsignal, from said recording medium upon reception of said command inputmaking a request for execution of a special reproduction process,wherein said portions reproduced from said recording medium each have alength equal to a predetermined set segment length and are separatedfrom each other by a predetermined interval segment length.
 18. Theinformation-signal process method according to claim 17, further havingthe step of detecting an important segment among segments included insaid information signal, whereby such a reproduction segment is set thatsaid reproduction segment has a length equal to said predetermined setsegment length and includes said detected important segment detected bysaid important-segment detection means.